Introduction to Learning and Behavior, 6th Ed. - Psychology Textbook

Introduction to Learning and Behavior Sixth Edition Russell A. Powell MacEwan University P. Lynne Honey MacEwan University Diane G. Symbaluk MacEwan University Australia • Brazil • Canada • Mexico • Singapore • United Kingdom • United States Introduction to Learning and Behavior, Sixth Edition Russell A. Powell, P. Lynne Honey, and Diane G. Symbaluk Senior Vice President, Higher Education & Skills Product: Erin Joyner Product Director: Laura Ross Product Manager: Cazzie Reyes Vendor Content Manager: Valarmathy Munuswamy, Lumina Datamatics, Ltd. © 2023, 2017, 2013 Cengage Learning ALL RIGHTS RESERVED. No part of this work covered by the copyright herein may be reproduced or distributed in any form or by any means, except as permitted by U.S. copyright law, without the prior written permission of the copyright owner. For product information and technology assistance, contact us at Cengage Customer & Sales Support, !-800-354-9706 or support.cengage.com. For permission to use material from this text or product, submit all requests online at www. copyright.com. Further permissions questions can be e-mailed to permissionrequest@cengage.com. Library of Congress Control Number: 2021923941 Product Assistant: Jessica Witczak ISBN: 978-0-357-65847-5 Manufacturing Planner: Ron Montgomery Cengage 200 Pier 4 Boulevard Boston, MA 02210 USA Digital Delivery Quality Partner: Allison Marion Inventory Analyst: Seth Cohn Intellectual Property Analyst: Deanna Ettinger Intellectual Property Project Manager: Ilakkiya Jayagopi Production Service: Lumina Datamatics, Ltd. Art Director and Cover Designer: Angela Sheehan Cover Image: Miriam Doerr Martin Frommherz/ Shutterstock.com Cengage Learning is a leading provider of customized learning solutions with employees residing in nearly 40 different countries and sales in more than 125 countries around the world. Find your local representative at www.cengage.com. Cengage is committed to creating inclusive and diverse products, including perspectives that represent the diversity of people across all backgrounds, identities, opinions, and experiences. If you have any feedback related to this, please contact us at Cengage Customer & Sales Support, !-800-354-9706 or support.cengage.com. To learn more about Cengage platforms and services, register or access your online learning solution, or purchase materials for your course, visit www.cengage.com. Printed in the United States of America Print Number: 01 Print Year: 2022 To the parents, mentors, and students who shaped our behavior so well as to make this book a reality. Brief Contents Preface xvii About the Authors xxv Chapter 1 Introduction 1 Chapter 2 Research Methods Chapter 3 Elicited Behaviors and Classical Conditioning Chapter 4 Classical Conditioning: Basic Phenomena and 51 Various Complexities Chapter 94 132 5 Classical Conditioning: Underlying Processes and Practical Applications 167 Chapter 6 Operant Conditioning: Introduction 209 Chapter 7 Schedules and Theories of Reinforcement Chapter 8 Extinction and Stimulus Control Chapter 9 Escape, Avoidance, and Punishment 330 Chapter 10 Choice, Matching, and Self-Control 368 254 291 Chapter 11 Observational Learning and Rule-Governed Behavior 412 Chapter 12 Biological Dispositions in Learning Chapter 13 Comparative Cognition Appendix 449 483 A Brief Guide to Behavior Self-Management Quick Quiz Answers 529 539 Glossary 549 References Index 561 595 v Contents Preface xvii About the Authors xxv Introduction 1 Chapter 1 AND FURTHERMORE: Notation for Conditioning Diagrams 1.1 Historical Background 8 1.2 Five Schools of Behaviorism 5 Aristotle: Empiricism and the Laws of Association 8 Descartes: Mind–Body Dualism and the Reflex 10 The British Empiricists 10 Structuralism: The Experimental Study of Human Consciousness 11 Functionalism: The Study of the Adaptive Mind 12 The Theory of Evolution: Humans as Animals 13 Behaviorism: The Study of Observable Behavior 15 AND FURTHERMORE: John B. Watson: Behaviorism’s Controversial Founder 16 18 Watson’s Methodological Behaviorism 18 AND FURTHERMORE: Deliberate Practice and Expert Performance Hull’s Neobehaviorism 22 Tolman’s Cognitive Behaviorism 24 AND FURTHERMORE: How to Read Graphs 27 Bandura’s Social Learning Theory 29 Skinner’s Radical Behaviorism 31 ADVICE FOR THE LOVELORN 36 AND FURTHERMORE: The Life of B.!F. Skinner 39 20 In Review 42 Further Exploration 43 Study Questions 43 Concept Review 44 Chapter Test 46 Answers to Chapter Test 49 vii viii Contents Chapter 2 Research Methods 2.1 Basic Terms and Definitions 52 Independent and Dependent Variables Functional Relationships 54 Stimulus and Response 54 Overt and Covert Behavior 55 Appetitive and Aversive Stimuli 56 Motivating Operations 57 Contiguity and Contingency 58 51 52 2.2 Measurement of Behavior 59 Behavioral Definitions 59 Recording Methods 60 Assessing Reliability 65 2.3 Research Designs 66 Descriptive Research 66 Experimental Research 68 ADVICE FOR THE LOVELORN 82 2.4 Use of Animals in Behavioral Research 83 AND FURTHERMORE: Cruel Starvation or a Healthy Diet: The Ethics of Food!Restriction 85 In Review 86 Further Exploration 87 Study Questions 87 Concept Review 88 Chapter Test 90 Answers to Chapter Test 93 Chapter 3 Elicited Behaviors and Classical Conditioning 94 3.1 Elicited Behaviors Reflexes 95 Fixed Action Patterns 95 97 3.2 Simple Mechanisms of Learning 99 Habituation and Sensitization 99 Opponent-Process Theory of Emotion 102 3.3 Classical Conditioning 108 ADVICE FOR THE LOVELORN 109 Pavlov’s Discovery of Classical Conditioning 110 Basic Procedure and Definitions 111 Appetitive and Aversive Conditioning 115 AND FURTHERMORE: Classical Conditioning and Interpersonal Attraction 119 Excitatory and Inhibitory Conditioning 119 Contents Temporal Factors in Conditioning Pseudoconditioning 124 120 In Review 125 Further Exploration 126 Study Questions 126 Concept Review 127 Chapter Test 128 Answers to Chapter Test 131 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities 132 4.1 Some Basic Conditioning Phenomena 133 Acquisition 133 Extinction and Related Phenomena 134 Stimulus Generalization and Discrimination Experimental Neurosis 140 138 4.2 Three Extensions to Classical Conditioning 143 Higher-Order Conditioning 143 Sensory Preconditioning 145 AND FURTHERMORE: When Celebrities Misbehave US Revaluation 149 146 4.3 Four Examples of Specificity in!Classical Conditioning 151 Overshadowing 151 Blocking 153 Occasion Setting 156 Latent Inhibition 158 ADVICE FOR THE LOVELORN 160 In Review 161 Further Exploration 161 Study Questions 161 Concept Review 162 Chapter Test 163 Answers to Chapter Test 165 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications 167 5.1 Underlying Processes in Classical Conditioning Stimulus-Substitution Theory 168 168 ix x Contents Preparatory-Response Theory (and the Compensatory-Response Model) AND FURTHERMORE: Conditioned Compensatory Responses and Drug Overdose 174 Rescorla–Wagner Theory 175 ADVICE FOR THE LOVELORN 179 169 5.2 Practical Applications of Classical Conditioning 180 Understanding Phobias 180 AND FURTHERMORE: On Searching for Little Albert: A Personal Account 184 Treating Phobias 191 AND FURTHERMORE: Was Sigmund Freud a Behavior Analyst? 197 Aversion Therapy for Problem Behaviors 200 Medical Applications of Classical Conditioning 201 In Review 204 Further Exploration 204 Study Questions 204 Concept Review 205 Chapter Test 206 Answers to Chapter Test 208 Chapter 6 Operant Conditioning: Introduction 209 6.1 Historical Background 211 Thorndike’s Law of Effect 211 Skinner’s Selection by Consequences 213 6.2 Operant Conditioning 215 Operant Behavior 216 Operant Consequences: Reinforcers and Punishers 218 Operant Antecedents: Discriminative Stimuli 221 6.3 Four Types of Contingencies 225 Positive Reinforcement 227 Negative Reinforcement 227 Positive Punishment 229 Negative Punishment 229 AND FURTHERMORE: Operant Contingencies: Tricky Examples and Common!Confusions 232 6.4 Positive Reinforcement: Further Distinctions 234 Immediate Versus Delayed Reinforcement 234 Primary and Secondary Reinforcers 235 Intrinsic and Extrinsic Reinforcement 238 AND FURTHERMORE: Positive Reinforcement of Artistic Appreciation Natural and Contrived Reinforcers 241 240 Contents Shaping 243 ADVICE FOR THE LOVELORN 245 AND FURTHERMORE: Training Ishmael 246 In Review 248 Further Exploration 248 Study Questions 249 Concept Review 249 Chapter Test 251 Answers to Chapter Test 253 Chapter 7 Schedules and Theories of!Reinforcement 254 7.1 Schedules of Reinforcement 255 7.2 Theories of Reinforcement 277 Continuous Versus Intermittent Schedules 255 Four Basic Intermittent Schedules 256 Other Simple Schedules of Reinforcement 265 Complex Schedules of Reinforcement 271 Drive Reduction Theory 277 The Premack Principle 278 Response Deprivation Hypothesis 280 Behavioral Bliss Point Approach 282 ADVICE FOR THE LOVELORN 283 In Review 284 Further Exploration 284 Study Questions 284 Concept Review 285 Chapter Test 287 Answers to Chapter Test 290 Chapter 8 Extinction and Stimulus Control 8.1 Extinction 292 Side Effects of Extinction 294 ADVICE FOR THE LOVELORN 297 Resistance to Extinction 298 Spontaneous Recovery 301 Differential Reinforcement of Other Behavior 302 AND FURTHERMORE: Extinction of Bedtime Tantrums in Young Children 304 291 xi xii Contents 8.2 Stimulus Control 304 Stimulus Generalization and Discrimination 305 The Peak Shift Effect 308 Multiple Schedules and Behavioral Contrast 311 Fading and Errorless Discrimination Learning 315 AND FURTHERMORE: St. Neots Margin 316 Stimulus Control: Additional Applications 319 AND FURTHERMORE: Edwin Guthrie: Stimulus Control for the Practical!Person 323 In Review 324 Further Exploration 324 Study Questions 324 Concept Review 325 Chapter Test 326 Answers to Chapter Test 329 Chapter 9 Escape, Avoidance, and!Punishment 330 9.1 Escape and Avoidance 331 Two-Process Theory of Avoidance 333 Avoidance Conditioning and Phobias 335 Avoidance Conditioning and Obsessive–Compulsive Disorder 338 AND FURTHERMORE: Approach-Avoidance Conflict (or “I Can’t Believe I Volunteered for This!”) 341 9.2 Punishment 342 Types of Punishment 342 Problems with the Use of Punishment 346 Benefits and the Effective Use of Punishment Theories of Punishment 352 9.3 Noncontingent Punishment 354 349 Learned Helplessness 354 Masserman’s Experimental Neurosis 356 ADVICE FOR THE LOVELORN 358 AND FURTHERMORE: Dissociative Identity Disorder: A Behavioral Perspective 359 In Review 361 Further Exploration 361 Study Questions 361 Concept Review 362 Chapter Test 363 Answers to Chapter Test 366 Contents Chapter 10 Choice, Matching, and Self-Control 368 10.1 Choice and Matching 369 Concurrent Schedules 369 The Matching Law 371 Deviations from Matching 375 AND FURTHERMORE: Basketball and the Matching Law Matching and Melioration 380 10.2 Self-Control 379 383 Skinner on Self-Control 384 Self-Control as a Temporal Issue 387 AND FURTHERMORE: B.!F. Skinner: The Master of Self-Control 387 Mischel’s Delay of Gratification Paradigm 391 The Ainslie–Rachlin Model of Self-Control 392 The Small-But-Cumulative Effects Model 400 AND FURTHERMORE: Self-Control: How Sweet It Is! Or Is It? 403 In Review 404 Further Exploration 405 Study Questions 405 Concept Review 406 Chapter Test 407 Answers to Chapter Test 410 Chapter 11 Observational Learning and Rule-Governed Behavior 412 11.1 Observational or Social Learning 413 Contagious Behavior and Stimulus Enhancement 414 Observational Learning in Classical Conditioning 416 Observational Learning in Operant Conditioning 418 AND FURTHERMORE: It’s An Acquired Taste. . . . 419 Imitation 422 AND FURTHERMORE: Can Animals Teach? 425 Social Learning and Aggression 427 11.2 Rule-Governed Behavior 431 AND FURTHERMORE: The Analysis of Verbal Behavior 431 Definitions and Characteristics 432 Some Disadvantages of Rule-Governed Behavior 434 AND FURTHERMORE: Psychopathology, Psychotherapy, and Rule-Governed Behavior 436 Personal Rules in Self-Regulation 437 ADVICE FOR THE LOVELORN 438 AND FURTHERMORE: Say–Do Correspondence and Willpower 440 xiii xiv Contents In Review 443 Further Exploration 444 Study Questions 444 Concept Review 445 Chapter Test 446 Answers to Chapter Test 448 Chapter 12 Biological Dispositions in Learning 449 12.1 Preparedness and Conditioning 450 Preparedness in Classical Conditioning 450 AND FURTHERMORE: Conditioned Food Preferences Preparedness in Operant Conditioning 457 12.2 Operant–Respondent Interactions Instinctive Drift 458 Sign Tracking 460 ADVICE FOR THE LOVELORN 457 458 463 12.3 Adjunctive Behavior 463 Basic Procedure and Defining Characteristics 464 Adjunctive Behavior in Humans 466 Adjunctive Behavior as Displacement Activity 467 12.4 Activity Anorexia 469 Basic Procedure and Defining Characteristics Comparisons with Anorexia Nervosa 470 Underlying Mechanisms 472 Clinical Implications 473 469 12.5 Behavior Systems Theory 474 AND FURTHERMORE: The Healthy Side of the Diet–Activity Connection 475 In Review 478 Further Exploration 478 Study Questions 478 Concept Review 479 Chapter Test 480 Answers to Chapter Test 482 Contents Chapter 13 Comparative Cognition 483 13.1 What Is Comparative Cognition? 13.2 Memory in Animals 486 484 How to Study Animal Memory 486 Memory in Food-Storing Birds 490 13.3 Can Animals Count? 492 The Case of Clever Hans 492 Beyond Clever Hans: Evidence for Numerical Ability in Animals 494 AND FURTHERMORE: When “Bird Brains” Outperform People 496 13.4 Category Learning and Relational Decisions 498 13.5 Concepts of Tools, Self, and Others 501 Making and Using Tools 501 Theory of Mind 504 Self-Awareness and Differentiating Self from Others Cooperation and Deception 508 13.6 Language 504 510 AND FURTHERMORE: A Border Collie Is Worth a Thousand Words Can Animals “Talk”? 515 Sign Language Experiments 516 Artificial Language Experiments 518 AND FURTHERMORE: Alex Speaks 521 ADVICE FOR THE LOVELORN 523 In Review 524 Further Exploration 524 Study Questions 524 Concept Review 525 Chapter Test 526 Answers to Chapter Test 528 Appendix A Brief Guide to Behavior Self-Management 529 Quick Quiz Answers 539 Glossary 549 References 561 Index 595 514 xv Preface “I wouldn’t do this to my budgie,” a student once muttered after a lecture in which I (the senior author of this text) had discussed the process of reinforcement. The student apparently saw the use of reinforcement as manipulative and reprehensible. I can’t remember how I responded (probably with something a bit more diplomatic than what follows), but I could have said that she actually does “this” to her pet bird all the time and is simply not aware of it. Moreover, because she’s not aware of it, she may be reinforcing her budgie’s behavior quite erratically, with the result that the two of them are having a much less satisfying relationship than they could be having. Unfortunately, this student’s negative reaction to behavioral principles of conditioning is not uncommon, and most instructors who teach such courses can probably recount similar instances. Thus, one goal of this text is to help convince students that conditioning is not a dangerous form of manipulation but rather a natural process that we do far better to understand and apply wisely than to ignore and apply carelessly. Another opinion sometimes voiced is that the principles of conditioning— many of which have been derived from research with animals—are largely irrelevant to important aspects of human behavior. After all, how can studies of lever-pressing rats or key-pecking pigeons say anything meaningful about what truly matters to us? This was the very conclusion that I (the senior author again) came to when, as an undergraduate, I first encountered a demonstration of operant conditioning in my introductory psychology class. We were shown a film in which pigeons were taught to peck a little plastic disk (which I later learned is called a “response key”) to earn food. The whole endeavor struck me as so completely artificial—not to mention mind-numbingly boring—that I couldn’t understand why anyone would waste their time on it. Little did I know that some years later I would find myself sitting in a pigeon lab, thrilled that I had been given an opportunity to study something so interesting and important! What I had learned in the interim was that (1) you have to be careful what you criticize (fate has a way of making us pay for our arrogance) and (2) many of the principles derived from conditioning experiments with animals are among the most useful principles ever discovered by psychologists. Thus, a second goal of this text is to help convince students that the principles derived from behavioral research are far xvii xviii Preface from irrelevant, and that they often have useful and provocative things to say about human behavior. An even more basic goal of this text is to provide students with a clear introduction to the basic principles of learning and behavior that is both accessible and engaging, especially for those who have had only limited exposure to them (such as in an introductory psychology course). Those students who later proceed to a higher-level course in the subject matter will then have a solid foundation on which to build. And students who do not proceed to a higher-level course will nevertheless have gained an appreciation for the behavioral perspective and learned much that may be of relevance to their everyday lives and future careers. Key Characteristics The following summarizes some key characteristics of this text: • It emphasizes basic principles of learning and behavior rather than theory. To the extent that theory is discussed, it is either because the theory itself has something meaningful and provocative to say about human behavior (e.g., melioration theory as discussed in Chapter 10) or because a general overview of certain theories (e.g., the Rescorla–Wagner theory, as presented in Chapter 5) can help prepare students for a more in-depth discussion of those theories in a higher-level course. • It attempts to strike a balance between basic research findings, many of which are derived from animal research, and the application of those findings to important and interesting aspects of human behavior. Although many texts make this claim, we feel that this text represents a truly concerted effort in that regard. Wherever possible, examples of research findings with rats or pigeons are juxtaposed with everyday examples of human behavior. And although some of the applications to humans are highly speculative, they nevertheless represent the type of speculation that behaviorists themselves often engage in and that many students find interesting and memorable. • Following the previous point, this text is especially innovative in the many examples given of the application of behavioral principles to understanding certain aspects of romantic relationships. In particular, scattered throughout the text are Advice for the Lovelorn columns in which hypothetical students are given behavioral-type advice concerning their relationship difficulties. Personal relationships are, of course, an important issue for many students, and students are often fascinated by the notion that behavioral principles may be helpful in understanding and resolving problematic relationships. These columns have thus proven to be an effective way to maintain students’ interest in the material, enhance their grasp of certain concepts, and provide them with a sense of what it means to “think like a behaviorist.” (Students are of course given due warning that Preface the advice in these columns is quite speculative and not to be taken too seriously.) • The text contains many interesting and thought-provoking topics not normally found in textbooks on learning and behavior. This includes such topics as what it means to have “willpower,” the controversy over the inadvertent creation of multiple personalities during psychotherapy, and how personal difficulties can contribute to a sense of “meaning” in life, all discussed from a behavioral perspective. Many of these topics are presented in special boxed inserts titled And Furthermore, which are intended to expand on material presented in the preceding section. • This text contains numerous pedagogical features that are designed to facilitate students’ ability to study and understand the material. These features are described later in the section on learning aids. Changes to the Sixth Edition Overall changes to this edition of the text include the following: • Numerous definitions and explanations have been modified to clarify certain concepts that students often have difficulty with (such as an expanded explanation in Chapter 1 on how to read graphs, a clearer distinction in Chapter 2 between speed and duration measures of behavior, and an example of how to use the matching law in Chapter 10 to calculate the distribution of behavior across specific schedules of reinforcement). • Most of the diagrams of classical and operant conditioning have been reformatted to include the abbreviated labels (e.g., NS, UR, etc.) on the same line as the stimuli and responses. For example, in previous editions, the conditioning trial for a standard classical conditioning procedure would be formatted like this: Metronome: Food Salivation NS US UR In this edition, it is formatted like this: Metronome (NS): Food (US) Salivation (UR) The advantage of placing it all on a single line is that students who are typing it out—such as on their laptop during a lecture or in an e-mail message to their instructor—will not have to fuss with trying to position the label beneath each element. • Each chapter now includes a list of Learning Objectives at the start of the chapter along with In Review and Further Exploration sections at the end of the chapter, the latter two replacing the Chapter Summary and Suggested Readings sections in previous editions. Additionally, the answers for all Quick Quiz items are now listed at the end of the textbook. These changes are more fully described in the Learning Aids section below. xix xx Preface Chapter-specific changes in this edition include the following: In Chapter 1, the term “countercontrol” has been omitted from the discussion of Skinner’s general approach to the issue of self-control. In previous editions, the term was broadly defined as the application of behavioral principles to counter negative influences of the environment on our behavior; however, the term is more commonly defined as the application of behavioral principles to counter efforts by other people to exert aversive social control over our behavior. Also in keeping with more common usage, the term behavior analysis is now defined as the general approach to the science (rather than the basic science) that grew out of radical behaviorism, which we then further divide into basic behavior analysis (also known as the experimental analysis of behavior) and applied behavior analysis. Changes to Chapter 2 include a discussion of quasi-independent variables as well as a brief description of a multi-element design (which instructors might wish to elaborate upon in their lectures). Chapter 4 also includes a discussion of the distinction between extinction and forgetting, which some students mistakenly assume are equivalent. The discussion of the Little Albert experiment in Chapter 5 includes a recent analysis of Watson’s (1923) film of the experiment which, contrary to Watson’s claim, provides little evidence of fear conditioning (Powell & Schmaltz, 2020). Chapter 5 also now includes a description of the use of virtual reality procedures for treating phobias. In Chapter 6, the And Furthermore insert that discusses “tricky examples” of operant contingencies now also discusses two additional sources of confusion for students. One is the mistaken assumption that any behavior that seems to occur automatically, such as a well-learned habit like stepping on the brakes when a traffic light turns red, is necessarily a classically conditioned (respondent) behavior rather than an operant behavior. The second is the mistaken notion that any consequence that stops the ongoing occurrence of an operant behavior—including, for example, the cessation of whining when a child is given an iPad that they have been whining for—is thereby a punisher (ignoring the fact that operant conditioning concerns the effect of a consequence on the future probability of a behavior). Along the same lines, the discussion of extinction in Chapter 8 and punishment in Chapter 9 draws attention to the fact that there are several other ways in which the frequency of a behavior can decrease—many students having a tendency to overapply the term extinction or punishment to any situation in which a behavior seems to weaken. Chapter 10 now includes a simplified step-by-step guide on how to calculate what the matching law predicts concerning the expected proportion of responses emitted on two concurrent VI schedules of reinforcement. It also includes a description of a phenomenon known as “revenge bedtime procrastination.” The Appendix, A Brief Guide to Behavior Self-Management, has been revised to strengthen its usefulness as a stand-alone guide for students wishing to get a head start on managing their own behavior, especially study behavior, as they proceed through the course. The introduction, in particular, has been Preface expanded to present a more complete behavioral model of self-control that, in our experience, students seem to find both enlightening and motivating. Additional changes to the Appendix include the listing of four specific selfmanagement tactics—the Pomodoro technique, the just-get-started tactic, the creation of a dedicated study space, and the Premack principle (as applied to self-reinforcement)—that students often report to be especially effective and may be useful starting points for self-experimentation. Learning Aids This text contains many pedagogical features designed to facilitate students’ reading and comprehension of the material. These include the following: • Opening Vignettes. Each chapter begins with a chapter outline, followed by either a quotation or a vignette related to the material presented in that chapter. The vignettes usually consist of a short fictional scenario illustrating a particular concept. The exact concept involved is not immediately revealed, however, thus encouraging students to actively ponder how the material they are reading may be related to the scenario. (An explanation of the concept each scenario is intended to illustrate can be found in the instructor’s manual.) • Learning Objectives. New to this edition, the start of each chapter now includes a brief set of learning objectives that serve to provide students with a brief overview (or roadmap) of the main concepts to be covered within that chapter. • Quick Quizzes. Scattered throughout each chapter are several fill-in-theblank quizzes. The purpose of these quizzes is to help students actively work with the material as they read it. Although an early reviewer of the first edition commented that such frequent quizzing might frustrate students by interrupting their reading, actual use of the material in class revealed quite the opposite. Students commented that the quizzes were extremely beneficial in helping them engage with the material. They especially appreciated quizzes that were embedded within sections that they perceived as quite technical, simply because the quizzes broke the material up into short chunks that they were better able to assimilate. Students therefore demanded more quizzes, not fewer, and the authors duly complied. Because the quiz items tend to be quite straightforward, with the answers easily checked by simply reviewing the relevant textbook material, past editions did not include the answers within the textbook. Students, however, have frequently indicated that they would very much prefer to have the answers provided within the text. We have therefore, new to this edition, provided an answer key to each of the quizzes at the back of the text. • In Review. New to this edition, the original Chapter Summaries have been replaced with an “In Review” section that provides students with xxi xxii Preface suggestions on how to create their own chapter summary. (A difficulty with the original Chapter Summaries was that students were too often tempted to skip reading the chapter and read only the chapter summary—which of course resulted in only a superficial understanding of the material and poorer performance on exams.) • Further Exploration. Also new to this edition, the original Suggested Readings section at the end of each chapter, which in our experience almost no students paid attention to, has been replaced with a Further Exploration section containing interesting links to Internet sites that illustrate or discuss certain concepts covered within the chapter. • Study Questions. A focused set of between 10 and 15 study questions is included at the end of each chapter. These study questions cover the most basic concepts discussed in that chapter. You can motivate students to answer these questions by informing them that some of the questions will appear as short-answer items on quizzes or exams. In fact, the senior author’s own strategy is to give students a random sample of four or five of these questions as a weekly chapter test, with students in each class being unaware of which questions they will be asked. • Concept Reviews. Each chapter is followed by a concept review, which lists all key terms and definitions in the chapter. These key terms and definitions are then reiterated in the glossary at the end of the text. • Chapter Tests. Each chapter ends with a chapter test, consisting mostly of fill-in-the-blank items. This test provides comprehensive coverage of the material presented in the chapter. It differs from the Quick Quizzes in that more items are of a conceptual, rather than factual or definitional, nature, thereby encouraging students to think more deeply about the material. These test items are numbered in random order so that students can immediately look up the answer to any particular item without having to worry about inadvertently seeing the answer to the next item. Instructor Resources Additional instructor resources for this product are available online. Instructor assets include an Instructor’s Manual, PowerPoint® slides, and a test bank powered by Cognero®. Sign up or sign in at www.cengage.com to search for and access this product and its online resources. Instructor’s Manual The Instructor’s Manual contains a chapter outline, discussion questions, activities, and additional materials, such as a set of annotated Internet resources where students can find information of interest. The Instructor’s Manual also contains a description of how the Advice for the Lovelorn column Preface can be adapted as a student assignment (along with additional examples of such columns that can be provided to students to facilitate their own efforts). Acknowledgments We wish to thank “Dr. Dee,” Ally McBeal (who was all the rage on TV when this text was first conceived), and all the other people (real and fictional) who inspired the Advice for the Lovelorn features and other aspects of this text. We also thank the people at Cengage Learning, including Laura Ross, Cazzie Reyes, Jessica Witczak, Sheila Moran, Angela Sheehan, Allison Marion, Deanna Ettinger, Valarmathy Munuswamy, and Ilakkiya Jayagopi, for their support in helping create this edition and for making the process of revising easier than it might otherwise have been. Finally, a special thanks to Dr. Suzanne E. MacDonald, who was a coauthor on early editions of the text and whose influence can still be seen within certain chapters. Russ Powell Lynne Honey Diane Symbaluk xxiii About the Authors Russell A. Powell Russ Powell earned his Ph.D. in psychology under the engaging mentorship of Frank Epling and David Pierce at the University of Alberta. As a veteran faculty member at MacEwan University, Russ has taught classes in learning and behavior for over 30 years using a variety of behaviorally inspired formats. He has also conducted and published research on a wide range of topics, including operant conditioning (choice behavior), dissociative identity disorder, sleep paralysis nightmares, self-regulation, and the history of psychology. He also helped identify the person now believed to have been Little Albert, the infant in whom Watson and Rayner (1920) famously (or infamously) attempted to condition a phobia of furry animals (e.g., Powell, Digdon, Harris, & Smithson, 2014). More recently, he coauthored a detailed analysis of Watson’s (1923) film of the Little Albert experiment, which strongly suggests that the fear conditioning procedure was largely unsuccessful and that the film clips were manipulated in such a way as to increase its value as a promotional device for Watson’s research endeavors (Powell & Schmaltz, 2021). P. Lynne Honey Lynne Honey—a self-described “evolutionary behaviorist”—completed a Ph.D. in experimental psychology in Jeff Galef’s lab at McMaster University, studying the role of social learning on alcohol consumption in rats. She has published a number of papers on this topic and considers social learning to be one of the most powerful adaptations available to our species and others. Dr. Honey joined the Department of Psychology at MacEwan University in 2003 because of its focus on teaching and student engagement. She currently conducts research on human social behavior in an evolutionary context, focusing on the influence of personality traits on social behaviors. She also studies the effectiveness of various teaching methods and has won a awards for innovation and excellence in teaching. xxv xxvi About the Authors Diane G. Symbaluk Diane Symbaluk received her Ph.D. in sociology from the University of Alberta in 1997, with a specialization in criminology and social psychology. She joined MacEwan University in 1996 to pursue her joint passion for teaching and research mentorship. Her publications include textbooks, journal articles, and more than 40 pedagogical resources (e.g., study guides, test banks, instructor manuals, and online resources). Her many contributions to student success through teaching and educational leadership have been recognized via Distinguished Teaching Awards and a 3M National Teaching Fellowship. Chapter 1 Introduction A review of Gerald Zuriff’s Behaviorism: A Conceptual Reconstruction (1985) . . . begins with a story about two behaviorists. They make love and then one of them says, “That was fine for you. How was it for me?” The reviewer, P.!N. Johnson-Laird, insists that [this story has a ring of truth about it]. Behaviorists are not supposed to have feelings, or at least to admit that they have them. Of the many ways in which behaviorism has been misunderstood for so many years, that is perhaps the commonest. . . . [In fact,] how people feel is often as important as what they do. B.!F. SKINNER, 1989, P. 3 Of all contemporary psychologists, B.!F. Skinner is perhaps the most honored and the most maligned, the most widely recognized and the most misrepresented, the most cited and the most misunderstood. A. CHARLES CATANIA, 1988, p. 3 Chapter Outline 1.1 Historical Background Aristotle: Empiricism and the Laws of Association Descartes: Mind–Body Dualism and the Reflex The British Empiricists Structuralism: The Experimental Study of Human Consciousness Functionalism: The Study of the Adaptive Mind The Theory of Evolution: Humans as Animals Behaviorism: The Study of Observable Behavior 1.2 Five Schools of Behaviorism Watson’s Methodological Behaviorism Hull’s Neobehaviorism Tolman’s Cognitive Behaviorism Bandura’s Social Learning Theory Skinner’s Radical Behaviorism Learning Objectives 1-1 Define learning. 1-2 Create a simple diagram of a classical conditioning procedure. 1-3 Outline earlier philosophical and scientific approaches that influenced the development of behaviorism. 1 2 Chapter 1 Introduction 1-4 Define empiricism as a philosophical approach to knowledge. 1-5 Describe similarities and differences between the structuralist and functionalist approaches to the study of psychology. 1-6 Outline the process of evolution by natural selection, including three key features. 1-7 Identify the researchers most strongly associated with the five presented forms of behaviorism. 1-8 Describe similarities and differences among the five presented forms of behaviorism. You may have already flipped through this book and noticed a lot of graphs and diagrams about experiments with rats and pigeons. Perhaps that seems terribly boring to you, and you might wonder how studies with “dumb animals” could be of any relevance to your life (unless you have a pet rat that you want to train, in which case you might view this book as very useful!). But what if the principles discovered in those experiments could help you improve your study habits, understand psychological disorders, and overcome your fear of spiders? In fact, what if those same principles could even help you improve your romantic relationships or be a better parent? Hmm, perhaps the studies are not so boring after all. Can the study of learning, and the experiments described in this book, really give you insight about those very personal topics? Let’s consider a few of these claims in more detail. Improving self-control and productivity. Do you too often find yourself watching online videos rather than studying? Or endlessly scrolling through social media rather than working out, as you had planned? In other words, do you often find yourself being sidetracked from your long-term goals by activities that are more immediately pleasurable? This book outlines the principles by which our environment affects our behavior, including behaviors involved in self-control. Chapter 10, in particular, provides a theoretical account of why self-control is often difficult for us to achieve, while the appendix, A Brief Guide to Behavior Self-Management, offers practical advice on how to improve self-control. In addition, we’ve included a series of Study Tips that offer specific advice on how to improve both the quality and quantity of studying that you do. The first one directly follows this section and offers research-based advice on how to effectively study a textbook. How to study a text is a useful skill that many students have never been taught, and we feel that a textbook about learning is a great place to learn about it! Additional information on improving study behavior, as well as other behaviors we might wish to control, can be found scattered throughout the text. Understanding psychological disorders. The principles of learning and behavior are important for understanding and treating a variety of disorders. For example, through a simple manipulation of a rat’s feeding schedule, the rat can be induced to stop eating and engage in extreme levels of exercise. In this book, you will learn how similar processes might account for the development of an eating disorder known as anorexia nervosa. Other sections describe the role of learning in our Introduction experiences of anxiety and depression as well as symptoms of posttraumatic stress disorder (PTSD) and dissociative identity disorder (DID). You’ll also learn about various behavioral therapies that have proven useful for treating those disorders. Overcoming fears. Most of us have fears that we would like to manage, even if they don’t reach the intensity associated with a phobia or another anxiety disorder. Whether you fear spiders, snakes, or exams, this textbook will provide you with insight into how these fears develop. You will learn how the principles of classical conditioning and negative reinforcement underlie many of our fears and anxieties, and how these same principles suggest effective treatments for alleviating these symptoms. Improving relationships with others. In this text, we often use relationship issues to illustrate basic principles of learning and behavior. Further, each chapter contains an Advice for the Lovelorn column, in which relationship problems are discussed from a behavioral perspective. Although the advice given is necessarily speculative—and as such should not be taken too seriously—these columns highlight the manner in which behavioral principles have the potential to enrich our understanding of human relationships. Raising children. Our students sometimes comment that “no one should be allowed to have children until they have taken a course like this.” This is obviously an extreme position! Quite a few of our students are already parents, and they’ve told us that this course provided them with an understanding of their successful parenting strategies as well as those that proved less successful. They are able to articulate why certain approaches work better than others and now interpret the behavior of their children in a new light. Managing workers. Did you ever have a job in which the boss’s primary approach to management seemed to be to threaten and punish workers if a mistake was ever made? How about a job in which the boss was highly supportive and far more likely to praise workers for doing a good job than chastise them for doing a poor job? Quite apart from which position you found less stressful, in which of them were you also a more effective worker, willing to go the extra mile even when the boss didn’t know about it—in other words, a real team player who takes pride in their work? If someday you end up managing others, what type of manager do you want to be? The principles outlined in this textbook, which have proven highly effective in the workplace, are quite arguably the basis for becoming an effective manager. In Chapter 9, in particular, you will learn why managers are so often seduced into using threats and punishment, often with little awareness that they are undermining efficiency, not increasing it. In general, a proper grounding in the basic principles of learning and behavior will help you understand why you behave the way you do and how your behavior can often be changed for the better. This knowledge can make you a better student, a better parent, a better worker, a better manager, and a better friend or partner. In a very real sense, the principles described in this text have the potential to enrich both your life and the lives of others—even though many of these principles have been derived from research with rats and pigeons! 3 4 Chapter 1 Introduction Study Tip: Many students are unaware of how best to study a textbook. Too often, for example, they simply read and reread the material in the hope that something will sink in. But research has shown that for most students, this method is largely ineffective. Better approaches involve the “testing effect,” in which testing your ability to recall the information you just read can greatly enhance your memory for it (e.g., Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013). One version of this approach is the 3R method of studying (McDaniel, Howard, & Einstein, 2009). The 3R strategy, which stands for read–recite–review, involves the following steps: 1. You first read a short section of material, such as the material under a particular subheading—though it could be shorter or longer than this depending on the difficulty or complexity of the material. 2. Once you finish reading, you then recite to yourself, without looking at the material, the main points in what you just read. Don’t worry about how well you do at this point—you may in fact find it quite difficult—just do the best you can. 3. After attempting to recite the material, then review the material by going back over it, paying particular attention to anything you missed in step 2. Research has shown that, for many students, this approach can significantly enhance learning. Unfortunately, students tend not to study this way because it is difficult and effortful, unaware of the fact that it is the effort that makes it effective. Many students also like to underline (or highlight) as they read, but they unfortunately do so in ways that contribute little to learning (Dunlosky et al., 2013). A frequent problem is that students underline too much information, both important and unimportant. An alternate approach, which we’ll call “selective underlining,” is to first read the paragraph without underlining; you instead simply search for the important points you will want to underline later. After finishing the paragraph, you then go back and underline just the important points. This means that you will often be underlining only parts of sentences rather than whole sentences. This process is slower and more effortful than the typical way students underline, but your recall for the information is likely to be much greater. The underlined passages also provide an excellent summary of the chapter for you to review or use as a basis for making study notes. As you no doubt noticed, the paragraph you just read provides an example of selective underlining (though you yourself may have chosen a somewhat different set of phrases). Another possibility you may wish to experiment with is to combine selective underlining with the 3R approach; for example, selectively underline each paragraph within a section, then when you finish the section try to recite the information you underlined. You can find plenty of how-to-study advice on the Internet—but be careful with it; the Internet contains a lot of advice, both good and bad—or at your campus counseling center. The important thing is to find the combination of study tactics that works best for you, which may be rather different from what works best for someone else. Of course, this advice is of little use if you tend to procrastinate and rarely study in the first place. In that case, you may wish to consult the appendix at the end of this text for advice on behavior self-management. Even if your instructor has not assigned the appendix as part of your course, you may find useful advice that could help decrease your tendency to procrastinate. Introduction 5 Let’s begin with a brief outline of what this textbook is about. Simply put, behavior is any activity of an organism that can be observed or somehow measured. As we discuss in Chapter 2, the activity may be internal or external and may or may not be visible to others. Learning can be defined as a relatively enduring change in behavior that results from some type of experience. For example, reading this text is an example of a behavior, and any enduring change in your behavior as a result of reading this text (e.g., a change in your ability to speak knowledgeably about the subject matter) is an example of learning. Note that the change in behavior does not have to be immediate, and in some circumstances might not become evident until long after the experience has occurred. Moreover, although we tend to emphasize change as an indicator of learning, sometimes what we are learning is when not to change, that is, when to persist with a behavior when it is prudent to do so. This text emphasizes two fundamental forms of learning: classical and operant conditioning. Although these will be discussed in more detail later, a brief description of each is useful at this point. At its most basic level, classical conditioning (also known as Pavlovian conditioning or respondent conditioning) is the process by which certain inborn (which can also be called innate, inherited, or unlearned) behaviors come to be elicited in new circumstances. The behaviors involved are often what the average person regards as reflexive or “involuntary,” such as sneezing in response to dust or salivating in response to food. A familiar example of classical conditioning, which is often presented in introductory psychology textbooks, is that of a dog learning to salivate in response to a bell that was previously followed by food. This process can be diagrammed as follows: Bell: Food ! Salivation Bell ! Salivation (Refer to the “Notation for Conditioning Diagrams” in the And Furthermore box.) And Furthermore Notation for Conditioning Diagrams In this text, you will encounter many diagrams of conditioning procedures. In these diagrams, a colon separating two events indicates that the events occur in sequence. For example, the term Bell: Food means that the sound of a bell is followed by the presentation of food. An arrow between two events also indicates that they occur in sequence but with an emphasis on the fact that the first event produces or causes the second. (If you type your notes, you should be able to create an arrow by typing two hyphens (-) followed by a greater-than (>) symbol.) For example, “Food ! Salivation” means that the presentation of food causes the dog to salivate. Thus, with respect to a standard classical conditioning procedure, the term: Bell: Food ! Salivation 6 Chapter 1 Introduction means that the bell is presented just before the food, and the food in turn causes the dog to salivate. This is followed by: Bell ! Salivation which indicates that the presentation of the bell itself now causes the dog to salivate (because of the bell’s previous pairing with food). For clarity, we will usually italicize the behavior that is being conditioned (which is often called the “target behavior”). In writing out your notes, however, you may find it easier to indicate the target behavior by underlining it. For example: Bell: Food ! Salivation Bell ! Salivation As you will learn in this text, classical conditioning underlies many of our emotional responses and contributes to the development of our likes and dislikes. It can also lead to the development of debilitating fears and powerful feelings of sexual attraction. In contrast to classical conditioning, operant conditioning (also known as instrumental conditioning) involves the strengthening or weakening of a behavior as a result of its consequences; that is, the behavior is more or less likely to occur in the future as a function of what followed it. The behaviors involved are often those that we perceive as being goal-directed, or “voluntary.” A common experimental example is that of a rat that has learned to press a lever (the behavior) to obtain food (the consequence), the result of which is a future increase in the rat’s tendency to press the lever. This can be diagrammed as follows: Lever press ! Food pellet Result: Future likelihood of lever pressing increases Because the lever press produced a food pellet, the rat is subsequently more likely to press the lever again. In other words, the consequence of the behavior (the food pellet) has served to strengthen future occurrences of that behavior. Many of the behaviors that concern us each day are motivated by such consequences: we hit the remote button to turn on a favorite television show, compliment a loved one because it produces a smile, and study diligently to obtain a passing grade. The consequences can be either immediate, as in the first two examples, or delayed, as in the last example—though, as we will later discuss, the effect of delayed consequences on behavior can involve certain complexities. Because of its importance for humans, operant conditioning is the type of learning most strongly emphasized in this text. Although the text concentrates on classical and operant conditioning, other types of behavioral processes are also discussed. For example, in observational learning, the act of observing someone else’s behavior facilitates the occurrence of similar behavior in oneself. Certain types of largely Introduction 7 inherited behavior patterns, such as fixed action patterns, are also discussed, as is the effect of inherited dispositions in either facilitating or inhibiting certain types of learning. Let’s begin, however, with a brief overview of the historical background to the study of learning and behavior. 1. The term behavior refers to any activity of an organism that can be o 2. 3. 4. 5. 6. 7. or somehow m , whereas the term learning refers to a relatively (temporary/enduring) [for these types of items, you can simply underline the correct answer] in behavior as a result of some type of . ex In conditioning, behaviors that the average person typically regards as (voluntary/involuntary) come to be elicited in new situations. In conditioning, a behavior produces some type of . consequence that makes the behavior more or less likely to occur in the f -directed” and Such behaviors are typically those that are regarded as “g which the average person often perceives as being (voluntary/involuntary) in nature. Feeling anxious as you enter a dentist’s office is an example of a behavior that conditioning. has most likely been learned through (Think: Is anxiety the type of behavior you perceive as reflexive and involuntary or as goal-directed and voluntary?) Speaking with a loud voice in a noisy environment so that others will be able to hear you is an example of a behavior that has most likely been learned through conditioning. According to the notational system to be used in this text, the term A: B means that event A (produces/is followed by) event B, and the term X ! Y means that event X (produces/is followed by) event Y. An alternative term for classical conditioning is (Pavlovian/respondent/ instrumental) conditioning and an alternative term for operant conditioning is (Pavlovian/respondent/instrumental) conditioning Quick Quiz A While reading the text, you will frequently encounter fill-in-the-blank quizzes. Students report that these quizzes greatly facilitate the task of reading by dividing the material into small chunks and by encouraging them to be actively involved with the material. Also, if you are using the 3R or underlining approach mentioned in the earlier section on study tips, the quizzes provide a way for you to assess how well these techniques are working for you. Note that many of the items contain helpful hints, usually in the form of the initial letter or two of the word that should be inserted into the blank. The answer keys for each quiz can be found at the end of the textbook. However, don’t quickly look up the answer to an item you’re having difficulty with. Research has shown that a certain amount of uncertainty can actually facilitate learning (Dunlosky et al., 2013; Karpicke & Blunt, 2011; Schmidt & Bjork, 1992). In other words, the struggle itself is a good learning experience, so you should instead make a real effort to answer each item first. 8 Chapter 1 Introduction 1.1 Historical Background Just as it is impossible to outline all of the experiences that have made you who you are, it is impossible to outline all of the historical events that have contributed to the modern-day study of learning and behavior. Some particularly important contributions, however, are discussed in this section. Aristotle: Empiricism and the Laws of Association Aristotle was a Greek philosopher who lived between 384 and 322 ".#. Aristotle’s teacher, Plato, believed that everything we know is inborn (which he conceived of as “residing in our soul”); thus, learning is simply a process of inner reflection to uncover the knowledge that already exists within. Aristotle, however, disagreed with Plato and argued that knowledge is not inborn but instead is acquired through experience. Aristotle’s disagreement with Plato is an early example of the classic debate between nativism and empiricism, or nature and nurture. The nativist (nature) perspective assumes that a person’s abilities and tendencies are largely inborn, whereas the empiricist (nurture) perspective assumes that a person’s abilities and tendencies are mostly learned. Plato is thus an early example of a nativist and Aristotle is an early example of an empiricist.1 Aristotle also suggested that ideas come to be connected or associated with each other via four laws of association (well, actually three, but he also hinted at a fourth that later philosophers expanded upon). 1. The Law of Similarity. According to this law, events that are similar to each other are readily associated. For example, cars and trucks are readily associated because they are similar in appearance (wheels, doors, headlights, etc.) and function (both carry passengers and materials along roadways). These similarities enable us to learn to view cars and trucks as instances of a larger category of objects known as automobiles. 2. The Law of Contrast. According to this law, events that are opposite from each other are readily associated. For example, on a word association test, the word black often brings to mind the word white, and the word tall often brings to mind the word short. Likewise, the sight of your unwashed car reminds you of how nice it would look if you washed it, and an evening of studying reminds you of how enjoyable it would be to spend the evening not studying. 1 In philosophy, the term empiricism usually refers to the notion that knowledge can be gained only through sensory experience rather than through heredity or by pure reasoning. In psychology, the term has a slightly altered meaning, which is that a certain behavior or ability is the result of experience rather than heredity. Thus, the notion that great musicians have inherited a “gift” for music is a nativist viewpoint, whereas the notion that almost anyone can become a great musician given the right kind of experiences is an empiricist viewpoint. But the word empiricism can also be used in a methodological sense to refer to the gathering of information through systematic observation and experimentation, as in “behavioral psychology is an empirical approach to the study of behavior.” 1.1 Historical Background 9 3. The Law of Contiguity. This law states that events that occur in close proximity to each other are readily associated (contiguity means “closeness”). For example, a child quickly learns to associate thunder and lightning because the sound of thunder soon follows the flash of lightning. Thunder and lightning are also perceived as coming from the same direction. Imagine how difficult it would be to associate thunder and lightning if the thunder occurred several minutes after the lightning flash and came from a different direction. 4. The Law of Frequency. In addition to the three preceding laws, Aristotle mentioned a supplement to the law of contiguity, which is that the more frequently two items occur together, the more strongly they are associated. You will more strongly associate a friend with a certain perfume the more frequently you smell that perfume upon meeting them. Likewise, you will more strongly associate a term (such as the law of frequency) with its definition the more frequently you practice saying that definition whenever you encounter the term (as when using flash cards to help memorize basic terminology). Aristotle’s laws of association are not merely of historical interest. As you will read later, the laws of contiguity and frequency are still considered important aspects of learning. After all, how well could a dog learn to salivate to the sound of a bell if the bell preceded the presentation of food by several minutes, or if there was only one pairing of bell and food? 2. 3. 4. 5. 6. 7. 8. , emphasizes the role of (learning/heredity); the empiricist position, as exemplified , emphasizes the role of (learning/ by the Greek philosopher A heredity). Nativist is to (nature/nurture) as empiricist is to (nature/nurture). (If you find analogy questions like this confusing, an Internet search will yield helpful information on how to handle such questions.) The law of states that we associate events that are opposite states that we associate to each other, whereas the law of events that occur close to each other. According to the law of , we easily associate events that , the more resemble each other. According to the law of often two events occur together, the stronger the association. Animals that have fur, four legs, a tail, and can bark are quickly perceived as belonging to the same species. This is an example of the law of . The fact that the words full and empty are easily associated with each other is an . example of the law of The more often one practices a particular move in wrestling, the more likely one is to perform that move in a real match. This is an example of the law of . After once encountering a snake in her garage, Lisa is now quite nervous each . time she enters the garage. This is an example of the law of This is also an example of (classical/operant) conditioning. Quick Quiz B 1. The nativist position, as exemplified by the Greek philosopher P Chapter 1 Introduction Descartes: Mind–Body Dualism and the Reflex René Descartes (1596–1650) is the French philosopher who wrote the famous line “I think, therefore I am.” Fortunately for psychology, this was not his only contribution. In Descartes’s time, many people assumed that human behavior was governed entirely by free will or “reason.” Descartes disputed this notion and proposed a dualistic model of human nature. On the one hand, he claimed, we have a body that functions like a machine René Descartes and produces involuntary, reflexive behaviors (1596–1650) in response to external stimulation (such as sneezing in response to dust). On the other hand, we have a mind that has free will and produces behaviors that we regard as voluntary (such as choosing what to eat for dinner). Thus, Descartes’s notion of mind–body dualism proposes that some human behaviors are reflexes that are automatically elicited by external stimulation, while other behaviors are freely chosen and controlled by the mind. Descartes also believed that only humans possess free will and that the behavior of nonhuman animals is entirely reflexive. Descartes’s dualistic view of human nature was a major step in the scientific study of learning and behavior because it suggested that at least some behaviors—namely, reflexive behaviors—are mechanistic and could therefore be scientifically investigated. It also suggested that the study of animal behavior might yield useful information about the reflexive aspects of human behavior. Bettmann/CORBIS 10 The British Empiricists Although Descartes believed that the human mind has free will, he also assumed, like Plato, that some of the ideas contained within it (e.g., the concepts of time and space) are inborn. By contrast, a group of British philosophers, known as the British empiricists, maintained that almost all knowledge is a function of experience. For example, one of the major proponents of British empiricism, John Locke (1632–1704), proposed that a newborn’s mind is a blank slate (in Latin, tabula rasa) upon which environmental experiences are written. The British empiricists also believed that the conscious mind is composed of a finite set of basic elements (specific colors, sounds, smells, etc.) that are combined through the principles of association into complex sensations and thought patterns—a sort of psychological version of the notion that all physical matter consists of various combinations of the basic elements. 1.1 Historical Background or ref type of behavior behavior: an inv type of behavior that that functions like a machine, and a v is (determined/freely chosen). By contrast, the behavior of animals was believed . to be entirely 2. The British , such as John , maintained that knowledge was largely a function of ex and that the mind of a newborn infant is a (in Latin) t (which means r ). 3. They also believed that the mind is composed of a finite set of basic that are combined through the principles of to form our conscious experiences. Structuralism: The Experimental Study of Human Consciousness The History Collection/Alamy Stock Photo The British empiricists did not conduct any experiments to test their notion that the mind is composed of basic elements; their conclusions were instead based upon logical reasoning and the subjective examination of their own conscious experience. Realizing the deficiencies in this approach, the German philosopher Wilhelm Wundt (1832–1920) proposed using the scientific method to investigate the issue. This approach was Edward B. Titchener strongly promoted by an American student (1867–1927) of his, Edward Titchener (1867–1927), and became known as structuralism. Structuralism is an approach to psychology that studies the basic elements of the conscious mind and how they are combined to create more complex experiences. Structuralists made great use of the method of introspection, in which the person attempts to accurately describe their conscious thoughts, emotions, and sensations. To get a feel for how difficult this is, try to describe your conscious experience as you listen to the ticking of a clock (and just saying, “I’m bored” doesn’t cut it). One thing you might report is that the ticks seem to have a certain rhythm, with a series of two or three clicks being clustered together. You might also report a slight feeling of tension (is it pleasant or unpleasant?) that builds or decreases during each series of ticks. As you can imagine, an accurate report of what we introspectively observe can be quite difficult. Although this approach to psychology died out by the early 1900s (for reasons described shortly), its emphasis on systematic observation helped establish psychology as a scientific discipline. More importantly, its extreme emphasis on conscious experience as the proper subject matter for psychology Quick Quiz C 1. Descartes’s dualistic model proposed that humans engage in two types of 11 12 Chapter 1 Introduction resulted in a great deal of frustration and dissatisfaction—which eventually led to the later establishment of a more objective approach to psychology, known as behaviorism. Functionalism: The Study of the Adaptive Mind Quick Quiz D Pictorial Press Ltd/Alamy Stock Photo William James (1842–1910), often regarded as the founder of American psychology, helped establish the approach to psychology known as functionalism. Functionalism assumes that the mind evolved to help us adapt to the world around us and that the focus of psychology should be the study of those adaptive processes. This proposition was partially derived from Darwin’s theory William James (1842–1910) of evolution, which proposes that adaptive characteristics that enable a species to survive and reproduce tend to increase in frequency across generations while nonadaptive characteristics tend to die out. Thus, according to a functionalist perspective, characteristics that are highly typical of a species, such as the characteristic of consciousness in humans, must have some type of adaptive value. Based on such reasoning, functionalists believed that psychologists should not study the structure of the mind but instead study the adaptive significance of the mind. Learning, as an adaptive process, was therefore a topic of great interest to the functionalists. Moreover, although functionalists still made use of introspection and still emphasized the analysis of conscious experience (in this manner, being similar to the structuralists), they were not opposed to the study of animal behavior. Like Darwin, they believed that humans evolved in the same manner as other animals and that much of what we learn from studying animals might therefore be of direct relevance to humans. Not surprisingly, two of the most important figures in the early history of behaviorism, E.!L. Thorndike (discussed in Chapter 6) and John B. Watson (discussed later in this chapter), were students of functionalist psychologists. 1. The st approach to psychology attempted to identify the basic elements of the mind. The primary research method used for accomplishing , which consists of the careful this was the method of i description of one’s internal (thoughts/feelings/sensations/all of these). 2. Those who adopted the (functionalist/structuralist) approach to psychology emphasized the adaptive processes of the mind and were thus very interested in . the study of l 3. The (functionalist/structuralist) approach viewed animal research as (relevant/ irrelevant) to the study of human behavior in that humans were assumed to have evolved in a (similar/dissimilar) way to other animals. 1.1 Historical Background 4. The functionalists were similar to the structuralists in that they emphasized experience and often used the method of the study of c . i 5. William James was a (functionalist/structuralist), and Edward Titchener was a . The Theory of Evolution: Humans as Animals Philip Gendreau/Bettmann/CORBIS As we have seen, the theory of evolution had a significant influence on the development of behaviorism, which continues today. Charles Darwin published the theory of evolution in 1859 in his book On the Origin of Species by Means of Natural Selection (often simply called The Origin of Species). It describes how species, including humans, change across generations in response to environmental pressures. The Charles Darwin basis of this theory is the principle of natural (1809–1882) selection, which is the concept that organisms that are capable of adapting to environmental pressures are more likely to reproduce and pass along their adaptive characteristics than those that cannot adapt. There are three main components to the principle of natural selection. The first is that traits vary, both within a species (e.g., some dogs are larger than other dogs) and between species (e.g., humans have a slower metabolism than hummingbirds). The second is that many traits are heritable, meaning that they have a genetic basis and can be inherited by offspring. The third component of natural selection is that organisms must compete for limited resources (e.g., some hyenas will go hungry, but the ones who hunt together will eat more than the ones who hunt alone). Now let us put all three ideas together. Some individuals will have more offspring than others, because their inherited traits provide an advantage in their particular environment. You may have heard the phrase “survival of the fittest” associated with evolution. Survival is important, but only insofar as survival allows for a reproductive advantage. If every member of a species lived to be hundreds of years old but none of their genes made it into the next generation, then their survival would have no impact on the evolution of their species. In other words, successful individuals are more likely to have offspring who, when they inherit the successful traits from their parents, are also more likely to survive and have offspring. As this process continues through succeeding generations, the proportion of individuals possessing the successful traits increases while the proportion of individuals possessing the unsuccessful traits decreases. Eventually, the changed population might differ so much from the original population that it becomes a new species. 13 14 Chapter 1 Introduction Quick Quiz E It should be noted that direct reproduction is not the only way to replicate your genes. We have lots of genes in common with other humans and more genes in common with our closest relatives. Copies of our genes will get into the next generation through our siblings, cousins, and other kin. This is one of the reasons why evolution doesn’t necessarily lead to “selfish” behaviors. Sometimes the most adaptive behavior is to cooperate, to protect someone else, or even to sacrifice your own interests for the benefit of others. Thus, an evolutionary adaptation is a helpful genetic trait that evolves as a result of natural selection. We usually think of such adaptations as physical characteristics (e.g., the trunk of an elephant), but adaptations can also be behaviors. For example, as you will learn in Chapter 3, if you inadvertently place your hand over a flame, a flexion response will cause you automatically to pull your hand away from the damaging fire even before you consciously feel pain. You can imagine how an inborn reflex like this would help an individual live long enough to reproduce, compared to an individual who lacked such reflexes. A particularly important evolutionary adaptation, which is the focus of this text, is the ability to learn. Although we need fast reflexes for some situations, the slower processes of learning and thinking are beneficial for other situations. From an evolutionary perspective, the ability to learn evolved because it conferred significant survival advantages on those who had this ability. Thus, trying to make a “nature versus nurture” distinction between learning and evolution is overly simplistic, because the ability to learn (nurture) is itself inherited (nature) and many of the processes of learning seem to be automatic or reflexive. In this text, you will learn about features of learning that are common across a variety of species, which suggests that the ancestors of these species faced similar environmental pressures that resulted in the evolution of similar features. Nevertheless, you will also learn about certain between-species differences in learning ability. As noted, Darwin’s theory of evolution had a strong effect on the early development of behaviorism, especially through its influence on the functionalist school of psychology out of which behaviorism developed. It continues to have an effect through the recent increased attention to genetic factors in learning and through the growth of “evolutionary psychology” as a major area of study. 1. The three main components to the theory of natural selection are: a. b. c. 2. To say that a trait is h means that it has a genetic basis and can be inherited by offspring. 3. The real driving force behind evolution is not actually survival, but rather the advantage held by individuals who possess adaptive traits. re 4. It is simplistic to assume that one can draw a clear distinction between and n , because the way we learn is itself n trait. an inh 1.1 Historical Background Behaviorism: The Study of Observable Behavior Bettmann/Getty Images In 1913, a flamboyant young psychologist by the name of John B. Watson published a paper titled “Psychology as the Behaviorist Views It.” In it, he lamented the lack of progress achieved by experimental psychologists up to that time, particularly the lack of findings that had any practical John B. Watson significance. A major difficulty, Watson (1878–1958) believed, was the then-current emphasis on the study of conscious experience, especially as promoted by the structuralists. In particular, the method of introspection was proving to be highly unreliable. Researchers frequently failed to replicate each other’s findings, which often led to bitter squabbles. Watson mockingly described the types of arguments that often ensued: If you fail to reproduce my findings, it is not due to some fault in your apparatus or in the control of your stimulus, but it is due to the fact that your introspection is untrained. . . . If you can’t observe 3–9 states of clearness in attention, your introspection is poor. If, on the other hand, a feeling seems reasonably clear to you, your introspection is again faulty. You are seeing too much. Feelings are never clear. (Watson, 1913, p. 163) The difficulty, of course, is that we are unable to directly observe another person’s thoughts and feelings. We therefore have to make an inference that the person’s verbal reports about those thoughts and feelings are accurate.2 It is also the case that many of the questions being tackled by the structuralists were essentially unanswerable, such as whether sound has the quality of “extension in space” and whether there is a difference in “texture” between an imagined perception of an object and the actual perception of the object (Watson, 1913, p. 164). In a very real sense, experimental psychology seemed to be drowning in a sea of vaguely perceived images and difficult-to-describe mental events. Moreover, the notion that the proper subject matter of psychology was the study of consciousness was so strongly entrenched that it affected even those who studied animal behavior. As Watson exclaimed, On this view, after having determined our animal’s ability to learn, the simplicity or complexity of its methods of learning, the effect of past habit 2 An inference is a supposition or guess based on logical deduction rather than on observation. For example, if you describe to me a dream you had last night, your report is based on your direct observation of a subjective experience. But if I accept that description (because there seems to be no reason for you to lie about it), I am making an inference that your report is accurate. Now suppose I interpret the dream as indicating that you have some unresolved, unconscious conflict, and you accept that interpretation as true. We are now both making an inference that this unconscious conflict exists, because neither you nor I have directly observed it. Needless to say, inferences about unconscious processes are even more problematic than inferences about conscious processes, because not even the person in whom the unconscious process exists is able to directly observe it. (One therefore needs to be very cautious about assuming that a behavior pattern is a result of unconscious processes.) 15 16 Chapter 1 Introduction upon present response . . . we should still feel that the task is unfinished and that the results are worthless, until we can interpret them by analogy in the light of consciousness. [In other words,] we feel forced to say something about the possible mental processes of the animal. (Watson, 1913, p. 160) Watson reasoned that the only solution to this dilemma was to make psychology a purely “objective science” based solely on the study of directly observable behavior and the environmental events that surround it. All reference to internal processes, such as thoughts and feelings, were to be stricken from analysis. By objectifying psychology in this manner, Watson hoped that psychology could then join the ranks of the natural sciences —biology, chemistry, and physics—which had traditionally emphasized the study of observable phenomena. In Watson’s now-classic words, Psychology as the behaviorist views it is a purely objective experimental branch of natural science. Its theoretical goal is the prediction and control of behavior. Introspection forms no essential part of its methods, nor is the scientific value of its data dependent upon the readiness with which they lend themselves to interpretation in terms of consciousness. (Watson, 1913, p. 154) Thus, behaviorism is a natural science approach to psychology that emphasizes the study of environmental influences on observable behavior. Watson also believed strongly in the value of animal research. In keeping with his functionalist background—in turn following from Darwin’s theory of evolution—he believed that the principles governing the behavior of nonhuman species might also be relevant to the behavior of humans. Thus, traditional behavioral research is often conducted using nonhuman animals, primarily rats and pigeons. As many of the examples in this text illustrate, the results obtained from such research are often highly applicable to human behavior. It is worth noting that Watson was not the first psychologist to recommend a more objective, natural science approach to psychology. He simply reflected a growing sentiment among many researchers at that time that such a move was necessary. Watson’s arguments, however, were the most clearly stated and therefore had a strong effect. Thus, while his 1913 paper (which later became known as the “Behaviorist Manifesto”) did not have an immediate impact, its influence slowly grew until, by the 1920s, the behaviorist revolution was well under way. (For a brief discussion of Watson’s personal life, see “John B. Watson: Behaviorism’s Controversial Founder” in the And Furthermore box.) And Furthermore John B. Watson: Behaviorism’s Controversial Founder John B. Watson was a charismatic and aggressive individual and as such was perhaps ideally suited for lifting psychology out of the mentalistic quagmire in which it had become immersed. Unfortunately, those same traits led to a life of conflict. The most 1.1 Historical Background 17 infamous story concerns the manner in which Watson was forced to resign from his university position. According to one version, he and a female student were caught conducting intimate experiments on human sexual responding, and he was forced to resign over the resultant scandal. There is, however, little evidence for this story (see Benjamin, Whitaker, Ramsey, & Zeve, 2007, for a description of how this rumor became established), and the real events appear to be as follows. In 1920, at the height of his academic career, Watson began an affair with Rosalie Rayner, a graduate student whose family was well connected and powerful. Catching wind of the affair, Watson’s wife entered Rayner’s room during a social visit to the Rayner family and stole the letters Watson had written to his young lover. Watson’s wife then filed for divorce and used the letters to help win a lucrative settlement. Meanwhile, the university ordered Watson to end his affair with Rayner. Watson refused and immediately tendered his resignation. Soon after, news of Watson’s divorce and of the affair found its way into the media, with one of Watson’s love letters even appearing in several newspapers. Watson’s academic career was ruined. Cast adrift, Watson married Rayner and obtained a job with a New York advertising firm. In his new position, he attempted to promote a more scientific approach to the discipline of advertising—though the extent to which he had any significant influence on the industry is questionable (Coon, 1994). He also continued to publish books and magazine articles promoting his behavioristic views. In fact, Watson was very much the pop psychologist of his era, much like the present-day Dr. Phil. Unfortunately, as with pop psychology today, some of his advice was based more on personal opinion than on well-established principles. For example, Watson believed that children should be trained to act like adults and even recommended giving them a handshake, rather than a hug or a kiss, when sending them to bed! In fact, the only time he ever showed affection to his own children was when his wife died in 1935. Teary eyed, Watson lightly put his arms around his children as he told them that their mother had passed away, then never again mentioned her name. Not surprisingly, his children retained bitter memories of their upbringing, and one son later died by suicide. It has been suggested that Watson had an underlying fear of emotions. In his love relationships (and he had numerous affairs throughout his life), he was extremely impulsive and amorous; yet in a group setting, he would reportedly quickly leave the room when the discussion turned to emotional issues. Thus, although Watson’s proposal to banish thoughts and feelings from psychology helped establish it as a more objective science, it may also have reflected some personal difficulties. In his later years, Watson became something of a recluse, living in the country and raising animals. He had always been fond of animals—sometimes claiming he preferred their company to that of humans—which may account for his early interest in animal research. Interestingly, he also admitted that he didn’t know enough about child-rearing to have been writing books and magazine articles on the topic. He died at the age of 80 in 1958. (If you would like to learn more about Watson, about whom there are many rumors—both true and false—Mechanical Man: John Broadus Watson and the Beginnings of Behaviorism by Buckley [1989] is a very readable biography. And Watson’s famous [or infamous] “Little Albert” experiment, in which he claimed to have conditioned a fear of furry animals in a 12-month-old infant, is discussed in Chapter 5.) Quick Quiz F 18 Chapter 1 Introduction 1. Watson noted that a major problem with the method of int was that the results obtained were often unreliable. 2. A basic problem with relying on someone’s report about their thoughts and that the report is feelings is that we are making an inf accurate. This term is defined in the footnote as a supposition or guess based on rather than direct o . logical d 3. The notion that the proper subject matter of psychology should be the study of consciousness was so strong that even the researchers who studied behavior felt compelled to make inferences about mental processes in their subjects. 4. Watson argued that psychology needed to become a n science (like biology, chemistry, and physics) based solely on the study of directly events. ob 1.2 Five Schools of Behaviorism Many people mistakenly believe that behaviorism is a monolithic entity, with Watson’s views being the same views held by other behaviorists. In fact, there are several schools of behaviorism, each based on a somewhat different set of assumptions about how best to study environmental influences on behavior. In this section, we describe five of these schools, beginning with Watson’s original brand of behaviorism, which is sometimes referred to as methodological behaviorism (e.g., O’Donohue & Ferguson, 2001).3 Watson’s Methodological Behaviorism One of the most extreme versions of behaviorism is the one originally proposed by Watson (1913). Methodological behaviorism (also known as classical behaviorism) asserts that, for methodological reasons, psychologists should study environmental influences only on those behaviors that can be directly observed (see Figure 1.1). Subjectively perceived activities, such as thinking and feeling, are methodologically too difficult to assess to be included in a scientific analysis of behavior. Such activities can be included for analysis only if they can, in some way, be directly measured. Watson, for example, hypothesized that thinking actually involves minute movements of 3 Be aware that the names of the different schools presented here are not standardized and can also overlap in various ways. For example, the term radical behaviorism was once applied to Watson’s approach to behaviorism; it is occasionally still used that way but is now more commonly reserved for Skinner’s approach (Schneider & Morris, 1987). And the term methodological behaviorism, which we’ve given to Watson’s approach, is sometimes applied to any approach to psychology that rejects the value of data gathered through introspection (i.e., a person’s reports about their thoughts and feelings), including some cognitive approaches. Similarly, the term neobehaviorism, which we’ve reserved for Hull’s brand of behaviorism, is often applied to the newer (“neo” means new), less restrictive schools of behaviorism that arose subsequent to Watson’s approach, including Tolman’s and Skinner’s versions (in which case, for example, Tolman’s cognitive behaviorism is also considered to be a type of neobehaviorism). 1.2 Five Schools of Behaviorism Figure 1.1 In methodological behaviorism, internal events, such as consciously perceived thoughts and feelings as well as unconscious drives and motives, are excluded from the analysis of behavior. Instead, one studies the direct relationship between changes in the environment and changes in observable behavior. Internal events Environmental events • Conscious thoughts • and feelings • Unconscious drives • and motives Observable behavior the vocal cords in the larynx—and he enjoyed goading his critics by referring to his own thoughts as “laryngeal activity” (Buckley, 1989). If this were true, and if such movements could be precisely measured, then the act of thinking could be subjected to scientific analysis. (As it turns out, laryngeal activity is not a reliable measure of thinking.) It is important to emphasize that Watson’s behavioristic proposal to ignore thoughts and feelings in scientific analysis was not simply an attempt to dehumanize people or to pretend that thoughts and feelings do not exist (whatever his own personal biases may have been); rather, it was in part a logical response to a crisis. If the discipline of psychology were to survive, it would need to break free from the extreme mentalism of the time and adopt a much different perspective. Watson’s behavioristic call to arms, though extreme, accomplished just that. From a theoretical perspective, Watson’s specific view of learning was rather mechanistic. Drawing from Pavlov’s work on classical conditioning, he came to believe that all learning involves the development of a simple connection between an environmental event (the “stimulus”) and a specific behavior (the “response”). Watson’s theory of learning is therefore regarded as a type of stimulus–response (S–R) theory, in which learning consists of a connection being formed between a specific stimulus and a specific response. Complex behavior is presumed to involve extremely long chains of these S–R connections. Over time, Watson also became something of an extremist regarding the nature–nurture issue. In his original 1913 article, he had emphasized the influence of both heredity and environment on behavior. In fact, he was one of the first individuals to systematically study innate behavior patterns in animals (he spent several strenuous summers engaged in field research with a type of seabird). Later, however, following extensive observations of human infants, he came to the conclusion that humans inherit only a few fundamental reflexes along with three basic emotions: love, rage, and fear. Everything else, he believed, is learned. This led Watson, in 1930, to make one of his most famous claims: Give me a dozen healthy infants, well-formed, and my own specified world to bring them up in and I’ll guarantee to take any one at random and train 19 20 Chapter 1 Introduction him to become any type of specialist I might select—doctor, lawyer, artist, merchant-chief, and, yes, even beggar-man and thief, regardless of his talents, penchants, tendencies, abilities, vocations, and race of his ancestors. (p. 104) Quick Quiz G Unfortunately, many textbooks quote only this passage and omit the very next sentence, which reads, “I am going beyond my facts, but so have the advocates of the contrary and they have been doing it for many thousands of years” (p. 104). And this was precisely Watson’s point: the supposition that a person’s abilities are largely inherited has been strongly promoted throughout history (and has often been used to justify acts of discrimination and racism). Watson was one of the first to issue a strong challenge to this assumption, arguing instead that there is at least as much evidence suggesting that human abilities are mostly learned. For this reason, Watson’s behavioral model became quite popular among the reformists of his day who were attempting to combat racism. (For recent evidence on the importance of learning versus inherited ability, see “Deliberate Practice and Expert Performance” in the And Furthermore box.) As we previously noted, many people mistakenly equate behaviorism with Watson’s rather extreme version. In fact, few behaviorists were this extreme; instead, they developed approaches that were considerably more moderate. One of the most influential of these was Hull’s neobehaviorism, which we discuss next. 1. Watson’s brand of behaviorism is called behaviorism. 2. According to this type of behaviorism, psychologists should study the influence of on directly o b . Subjectively the e experienced internal events, such as thoughts and feelings, should be (excluded from/included in) the study of behavior. 3. Watson proposed a(n) – theory of learning, which hypothesizes that learning involves the formation of a direct and a r . connection between a st 4. In his 1913 article on behaviorism, Watson emphasized the influence of (heredity/ environment/both) in the development of behavior. In his later theorizing, he downplayed in the development of human behavior. the role of 5. In his later theorizing, Watson proposed that humans inherit (many/a few) basic , , reflexes, along with three basic emotions: . and And Furthermore Deliberate Practice and Expert Performance Watson’s emphasis on the importance of nurture over nature in determining human behavior is often viewed with skepticism. This is especially the case when it comes to behaviors that are indicative of exceptional ability. Most people, including many psychologists (e.g., Gardner, 1993), assume that unless a person is born with a certain amount of talent, there are limits in how far they will be able to progress in a particular endeavor. Indeed, the notion that a basketball star like Michael Jordan, a brilliant scientist like Marie Curie, or a musical genius 1.2 Five Schools of Behaviorism 21 like Wolfgang Amadeus Mozart is to a large extent born, and not made, is part of the mystique surrounding these individuals. But consider the following: • Expert performers in almost all fields of endeavor, ranging from music to athletics to chess, require years of intensive training before achieving a high level of performance. Even Mozart, who started composing at age four, did not compose world-class music until his late teens. Mozart’s father was also a professional musician who published the first book on violin instruction and provided his children with intensive musical training from an early age. (Mozart’s reputation has also benefitted from certain dubious claims: for example, the notion that he could compose entire works in memory and then write them down with little or no editing is based on a single passage in a supposed letter of his that is now believed to be a forgery [Colvin, 2008]). • Susan, Sophia, and Judit Polgár were chess prodigies in the 1970s and 1980s. Their father, László Polgár, is a Hungarian educator who set out to systematically train his three daughters to become expert chess players. Starting when they were toddlers, the girls were homeschooled, and their education was centered around chess. Judit could beat her father by the age of five, and all three were consistently winning tournaments before they reached their teens. The sisters all achieved high rankings in international chess: Susan and Judit achieved the rank of Grandmaster, and Judit became the youngest Grandmaster ever at the age of 15 (Flora, 2005). • The superlative abilities shown by experts are almost always specific to their field of endeavor. For example, chess experts have the ability to memorize the exact positions of all the chess pieces in a game after only a few seconds’ glance at the chessboard. But they perform no better than non-chess players at memorizing chess pieces randomly distributed around the board in a non-game pattern. • Almost all of the remarkable feats displayed by savants—individuals of low intellectual ability who nevertheless possess some remarkable skill—have been taught to people with average or typical abilities. For example, the ability of some savants to name the day of the week for any arbitrary date (e.g., “What day of the week was June 30, 1854?”) has been duplicated by randomly selected college students after only a few weeks of training. Based on findings such as these, Ericsson, Krampe, and Tesch-Römer (1993; see also Ericsson & Charness, 1994; Ericsson, 2009) argued that the most critical factor in determining expert performance is not innate ability but deliberate practice. Deliberate practice is practice that is not inherently enjoyable; it instead involves intense concentration and considerable effort with a view toward improving one’s performance. More than any other variable, the accumulated amount of deliberate practice in an activity is strongly predictive of an individual’s level of performance. Because deliberate practice is so effortful, the amount that can be tolerated each day is necessarily limited. For this reason, elite performers often practice about four hours per day. Ericsson et al. (1993), for example, found that the best violin students engaged in solitary practice (which was judged to be the most important type of practice) for approximately 3.5 hours per day, spread out across two to three sessions, each session lasting an average of 80 minutes. Note that this did not include time spent receiving instruction, giving performances, or playing for enjoyment. The students also devoted about 3.5 hours a day to rest and recreation and obtained more than average amounts of sleep. Top-level performers in intellectual pursuits display similar characteristics. Novelists typically write for about three to four hours each day, usually in the morning. Eminent scientists likewise write for a few hours each morning—the writing of articles arguably being the most important activity determining their success—and then devote the rest of the day to other duties. B. F. Skinner is especially instructive 22 Chapter 1 Introduction Deane Keller, Dr. Clark Leonard Hull (1884–1952), M. A. (Hon.) 1929 Yale University Art Gallery, Gift of Colleagues, friends, and students of the sitter in this regard. In his later life, he would rise at midnight and write for one hour, then rise again at 5:00 A.M. and write for another two hours. The remainder of the morning was devoted to correspondence and other professional tasks, while much of the afternoon was devoted to leisure activities such as tinkering in his workshop and listening to music. He deliberately resisted any urge to engage in serious writing at other times of the day, feeling that this often resulted in poor-quality writing the next morning. However, he more than compensated for the limited amount of writing he did each day by the consistency with which he wrote, resulting in a steady stream of influential articles and books throughout his career (Bjork, 1993). Ericsson et al. (1993) do not completely discount the role of heredity in expert performance. For example, heredity might well affect the extent to which one becomes interested in an endeavor as well as one’s ability to endure the hard work needed to become a top performer. More recently, some researchers (e.g., Hambrick, Macnamara, & Oswald, 2020) have argued that Ericsson and colleagues have underestimated the influence of heredity in various types of elite performance, including musical ability (Mosing, Peretz, & Ullén, 2017) and athletics (Schutte, Nederend, Hudziak, Bartels, & de Geus, 2016; but see Ericsson & Harwell, 2019, for their side of the debate). Nevertheless, it is now generally accepted that deliberate practice can have profound effects on performance and plays a key role in the development of expertise. We should, therefore, not be too quick to discount our ability to acquire a certain skill. Although most of us might not have the desire, time, or (perhaps) genetic makeup to become elite athletes, exceptional musicians, or top-level mathematicians, this in no way rules out the possibility of becoming better tennis players, learning to play the guitar, and increasing our proficiency in math. Hull’s Neobehaviorism A major challenge to methodological behaviorism came from Clark Hull (1884–1952), who claimed that Watson’s rejection of unobservable events was scientifically unsound. Hull noted that both physicists and chemists make inferences about events they have never directly observed but that can Clark L. Hull nevertheless be operationalized (i.e., (1884–1952) defined in such a way that they can be measured). For example, gravity cannot be directly observed, but its effect on falling objects can be precisely measured. Hull believed that it might likewise be useful for psychologists to infer the existence of internal events that might mediate (form a connection) between the environment and behavior. The mediating events that Hull incorporated into his theory consisted largely of physiological-type reactions, for example, a “hunger drive” that can be operationalized as number of hours of food deprivation. Such mediating events are formally called intervening variables, meaning that they intervene between a cause (such as food deprivation) and an effect (such 1.2 Five Schools of Behaviorism Figure 1.2 In Hull’s neobehaviorism, theorists make use of intervening variables in the form of hypothesized physiological processes to help explain the relationship between the environment and behavior. Internal events (intervening variables) Environmental events Internal physiological processes, such as hunger and fatigue Observable behavior as speed of running toward food). Thus, Hull’s neobehaviorism (sometimes also called deductive behaviorism) uses intervening variables, in the form of hypothesized physiological processes, to help explain behavior (see Figure 1.2). It is important to note that Hull’s use of intervening variables did not mean that he advocated a return to mentalism. Like Watson, he strongly opposed the use of introspection as a scientific tool, believing that subjective experiences are too vague and unreliable to be of much use. Thus, whether the organism actually experienced a feeling of hunger was of no concern to him. What did concern him was whether the concept of hunger, as defined in some measurable way (such as number of hours of food deprivation), was scientifically useful and led to testable hypotheses. Hull’s theory was also an S–R theory because it assumed that learning consists of the establishment of connections between specific stimuli and specific responses. Thus, like Watson, he viewed behavior in a very mechanistic, stimulus–response fashion. Lest this seem dehumanizing, recognize that it is not far removed from some modern-day cognitive approaches, which view humans as analogous to computers that process bits of information from the environment (input) to produce responses (output). This is actually quite similar to Hull’s model of behavior. In fact, some versions of modern-day cognitive psychology can even be considered outgrowths of Hull’s neobehaviorism.4 Hull was the most influential experimental psychologist of the 1940s and 1950s. Unfortunately, many aspects of his theory (which are beyond the scope of this text) were very difficult to test. The theory was also highly mathematical and grew increasingly complex as equations were expanded and modified. Some of these modifications were forced on Hull by his critics, the most famous of whom was Edward C. Tolman. (For an overview of Hull’s theory, as well as to gain a sense of its complexity, see Hull, 1943.) 4 Interestingly, people seem less critical of the cognitive information-processing approach to psychology, which draws an analogy between humans and computers, than they are of the traditional behavioral approach, which draws an analogy between humans and animals such as rats. Perhaps this is because we are impressed by the ability of computers to perform certain humanlike tasks (e.g., play chess) and insulted by the notion that humans and rats have anything in common. Yet, outside their specialized abilities, computers are quite inferior to rats. Imagine, for example, that a man, a rat, and a computer are washed up on a deserted island. To the extent that the man emulates the rat (if he is capable of it), he will likely survive; to the extent that he emulates the computer, he will sit on the beach and rot. Rats have a marvelous ability to learn and adapt; presentday computers do not. Fortunately for us, humans are far more ratlike than computerlike. 23 Chapter 1 Introduction 1. Hull believed that it might be useful to incorporate internal events into one’s by defining them in theorizing as long as they can be op such a way that they can be measured. 2. In Hull’s approach, the internal events he included were hypothetical processes. ph 3. Such internal events are called i variables in that they between the environment and are presumed to m behavior. 4. Hull’s theory was a(n) theory in that it assumed that the process of learning involves the creation of connections between specific and specific r . s The Drs. Nicholas and Dorothy Cummings Center for the History of Psychology, The University of Akron. Quick Quiz H 24 Tolman’s Cognitive Behaviorism Hull’s S–R theory of learning is often categorized as a “molecular” theory insofar as it assumed that specific S–R connections are the building blocks of behavior in the same way that molecules are the building blocks of matter. Edward Tolman (1886–1959) disagreed with this approach and believed that it would be more useful to analyze behavior on a “molar” (i.e., broader) level. For example, he felt that we can understand a rat’s behavior in a maze more accurately as a goal-directed Edward C. Tolman attempt to obtain food than as a long chain (1886–1959) of discrete stimulus–response connections that, in machinelike fashion, lead to food (e.g., Tolman, 1932). This molar approach to learning is similar to the gestalt approach to perception (Kohler, 1947), from which Tolman drew much of his inspiration. To the gestalt psychologists, perception is not simply the summation of different bits of conscious experience but is instead a “holistic” process resulting in an organized, coherent, perceptual experience. We perceive a house as more than just a combination of bricks and boards: it is bricks and boards plus something more. As the classic gestalt saying goes, “the whole is more than the sum of the parts.” Similarly, for Tolman, behavior is more than just a chain of discrete responses attached to discrete stimuli. It is instead an overall pattern of behavior directed toward particular outcomes and can be properly analyzed only on that level. Although Tolman disagreed with much of Hull’s theorizing, he did agree that intervening variables may be useful in a theory of learning (in fact, it was Tolman who first suggested this). However, while Hull’s intervening variables were often physiological-type processes like hunger and fatigue, Tolman’s were more mentalistic in nature. The Tolmanian rat, as well as the Tolmanian person, was not simply motivated by drives and habits but also had “expectations” and “hypotheses.” Thus, Tolman’s cognitive behaviorism (sometimes called purposive behaviorism) uses intervening variables, usually 1.2 Five Schools of Behaviorism in the form of hypothesized cognitive processes, to help explain behavior (see Figure!1.3). Tolman’s (1948) most famous intervening variable is the cognitive map, which is a mental representation of one’s spatial surroundings. Evidence for this concept was provided by a study on “latent learning” by Tolman and Honzik (1930). This experiment was conducted in an attempt to disprove Hull’s notion that behavior must be rewarded for learning to take place; that is, in the absence of some type of reward, nothing can be learned. To test this notion, Tolman and Honzik trained three groups of rats on a complex maze task (see Figure 1.4). The rats in a continuous-reward group always found food when they reached the goal box, but the rats in the two other groups Figure 1.3 In Tolman’s cognitive behaviorism, theorists make use of intervening variables in the form of hypothesized cognitive processes to help explain the relationship between environment and behavior. Internal events (intervening variables) Environmental events Internal cognitive processes, such as expectations and hypotheses Observable behavior Figure 1.4 Maze used by Tolman and Honzik (1930) in their study of latent learning. (Adapted from Elliott, M. H. (1928). The effect of change of reward on the maze performance of rats. University of California Publications in Psychology, 4.) Curtains Gates Start Food 25 Chapter 1 Introduction found no food when they reached the goal box (they were simply removed from the maze and then fed several hours later). Training proceeded at the rate of one trial per day for 10 consecutive days. As expected, the rewarded group learned to run quickly to the goal box, whereas the two nonrewarded groups took much longer to do so. Following this initial phase of training, on day 11 the rats in one of the nonrewarded groups also began receiving food when they reached the goal box. According to Hull, the rats in that group should only then have started to learn their way through the maze, which would have been demonstrated by a gradual improvement in their performance. What Tolman and Honzik found instead was a dramatic improvement in the rats’ performance on the very next trial (see Figure 1.5; see also “How to Read Graphs” in the And Furthermore box). Figure 1.5 Errors made by the different groups of rats in Tolman and Honzik’s (1930) latent learning experiment. The vertical axis represents the average number of wrong turns the rats in each group made before reaching the goal box. Group NR are those rats that never received a reward for reaching the goal box. Group R are those rats that always received a reward for reaching the goal box. Group NR-R received no reward for the first 10 days of the study, then began receiving a reward on day 11. Note that this group was run for a few days longer than the other two groups to see if there would be any additional change in their performance. (Adapted from Tolman, E.!C., & Honzik, C.!H. (1930). Degrees of hunger, reward and nonreward, and maze learning in rats. University of California Publications in Psychology, 4, 241–275.) 32 30 28 26 24 Errors 26 Group NR-R: no reward Group NR-R: reward 22 20 18 16 14 12 10 8 6 4 2 NR R NR-R 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 Days 1.2 Five Schools of Behaviorism 27 Tolman interpreted these results as indicating that the initially nonrewarded rats had in fact learned the maze during the first 10 trials of the experiment and that they had learned it at least as well as the group that had been receiving food. He would later interpret these findings as indicating that And Furthermore How to Read Graphs Graphs are a concise way of conveying information, but it can take some practice to interpret them. Let’s use Figure 1.5, which depicts errors made by rats in Tolman and Honzik’s experiment, as an example. This graph shares many features with other graphs that will be used throughout this textbook. It has two axes: the horizontal or x-axis, which is formally called the abscissa, and the vertical or y-axis, which is formally called the ordinate. The vertical axis is usually a measure of the target behavior in which we are interested; in Figure 1.5, this is the number of errors the rats made while running through a maze. The horizontal axis usually indicates some aspect of the experimental manipulation; in this case, the days on which the rats ran through the maze. The broken line between days 10 and 11 indicates that there was a change in conditions at this time, which is described by the labels on each side of the broken line (namely, that group NR-R switched from receiving no reward to receiving a reward). The three lines within the graph therefore indicate the average number of errors made by each group of rats on each day of the experiment. An important component of any graph is the figure caption, which is the brief description of the graph. As you read the figure caption, use your finger or cursor to point out each part of the explanation. For example, the caption for Figure 1.5 indicates that Group R are those rats that always received a reward for reaching the goal box. Drag your finger or cursor along the dotted line for Group R, and you will note the steady decrease in the number of errors (from an average of 28 to an average of 8) from Day 1 of the study to Day 17. Use your fingers or cursor to note that Group R made very few errors compared to group NR, who never got rewards for reaching the goal. Now follow the line for the NR-R group, noting a bit of a decrease in the first 10 days and then a sharp decrease in errors starting on Day 11. This sudden drop corresponds with the dotted line down the middle of the graph that indicates when that group started getting a reward for reaching the goal. The different paths for the three lines represent the different outcomes for the three groups. Not only are there differences on the last day of the study, but there are differences in the overall shapes of the paths. Learning to read graphs—and to create them—is a skill that you will use in a variety of classes and one that will help you interpret and verify results wherever you encounter them. If you are confused about data in a book, newspaper, or Web site, use the available information to create your own graph. You will give yourself another perspective on the data that could help you to understand it more completely. Chapter 1 Introduction the rats had developed a “cognitive map” of the maze during the early trials (Tolman, 1948), which became apparent only when they began to receive food in the goal box. Thus, this experiment is regarded as a classic demonstration of latent learning, in which learning can occur despite the absence of any observable indication of learning at that time and only becomes apparent at a later time. The experiment is also regarded as demonstrating the distinction between learning and performance, because learning was apparently taking place even when the subjects showed no evidence of learning in their performance at that time. Although Tolman believed that it was useful to incorporate cognitive variables into his theory, he remained in many ways a standard behaviorist. Like Hull and Watson, he believed that introspective reports of thoughts and feelings are so unreliable as to be of little scientific value. He maintained that his own theoretical inferences about cognitive processes were based entirely on direct observations of behavior and were thus objectively based. Tolman once even apologized for the “shameful necessity” of having to discuss conscious experience in a textbook he was writing (Tolman, 1932)—a reflection perhaps of how frustrated psychologists had been by the old introspectionist approach. Like other behaviorists, Tolman also believed strongly in the usefulness of animal research for discovering basic processes of learning, and almost all of his research was conducted with rats. Much of Tolman’s research was directly aimed at challenging Hull’s theory of learning. Hull responded by modifying his theory, in increasingly Pat Byrnes/The New Yorker Collection/The Cartoon Bank 28 1.2 Five Schools of Behaviorism 29 complex ways, to account for many of Tolman’s findings. 5 As a result, Tolman’s cognitive behaviorism never achieved the same popularity during his lifetime as Hull’s neobehavioral approach. With the growth of cognitive psychology, however, many of Tolman’s research methods and concepts have been adopted by modern researchers. Cognitive behaviorism is now a flourishing field of study, and the study of cognitive processes in nonhuman animals is now known as “animal cognition” or “comparative cognition” (which is the focus of Chapter 13). 2. 3. 4. 5. behaviorism because it uses mentalistic concepts, such as “expectations,” to explain behavior. This behaviorism. approach is also sometimes called p A c m is an internal representation of one’s surroundings. The experiment by Tolman and Honzik (1930) has traditionally been regarded learning, in which learning as a demonstration of appears to take place in the absence of any reward. The experiment has also been regarded as a demonstration of the distinction between learning and . Like many behaviorists, Tolman believed that introspectively observed thoughts and feelings are (useless/useful) in the analysis of behavior. As well, almost all of as subjects. Tolman’s research was conducted using The modern-day study of cognitive processes in nonhuman animals is known as c or com a . c Bandura’s Social Learning Theory If Tolman’s use of cognitive concepts seems to represent a partial return to mentalism, Albert Bandura’s social learning theory is an even stronger step in that direction. The roots of social learning theory can be partly traced to Hull’s neobehaviorism in that Bandura had considerable exposure to Hullian theorists during his graduate training. In fact, the term social learning theory was first used by followers of Hull who were attempting to apply 5 Roughly speaking, Hull (1943) tried to account for the results of Tolman and Honzik’s latent learning experiment by hypothesizing that the rats in the nonrewarded conditions found the mere act of being removed from the maze when they reached the empty goal box to be slightly rewarding. This slight reward was sufficient to ensure that the rats learned the pathway to the goal box but not sufficient to motivate them to greatly reduce the number of errors they were making. Only when the incentive to get to the goal box was increased by the availability of food did the number of errors drop significantly and the degree of learning become evident. Alternatively, Jensen (2006) has recently pointed to evidence indicating that the rats in the nonrewarded group may have found the act of entering a blind alley punishing, thereby reducing their tendency to make incorrect responses. This would again argue against interpreting these particular results as strong evidence of latent learning. Quick Quiz I 1. Tolman’s approach is known as Chapter 1 Introduction Hullian concepts to human social behavior, particularly the process of imitation (Miller & Dollard, 1941). Bandura was very much interested in imitation, which he referred to as observational learning, and he eventually became the dominant researcher in the field. His most famous investigations concern the effect of observational learning on aggressive behavior (Bandura, 1973). Although Bandura’s interests were partially influenced by Hullian psychologists, his interpretation of the learning process is more closely aligned with Tolman. Like Tolman, Albert Bandura Bandura focuses on broad behavior patterns (b. 1925) (i.e., he uses a molar approach) and strongly emphasizes the distinction between learning and performance. Unlike Tolman, however, he views internal events as more than just theoretically useful; they are seen as actual events occurring within us that strongly influence our behavior. This means that, unlike the other behaviorists we have discussed, Bandura does not dismiss the value of introspectively observed thoughts and feelings in explaining behavior. Thus, social learning theory (also called cognitive social learning theory or social cognitive theory) is an approach to psychology that strongly emphasizes the importance of observational learning and cognitive variables in explaining human behavior (Bandura, 1977, 1997). Bandura also has a distinct view of determinism (the notion that each behavior has a cause). Reciprocal determinism is the assumption that environmental events, observable behavior, and “person variables” (a!person’s thoughts and feelings) reciprocally influence each other (see Figure 1.6). Thus, according to Bandura, how we think and behave can influence our environment—through the environments we choose to be in as well as how we perceive those environments—as much as the environment influences how we think and behave. And behavior can influence our thoughts and feelings (“I’m stammering, therefore I must be anxious”) as much as our thoughts and feelings can influence our behavior. This Linda A. Cicero/Stanford News Service 30 Figure 1.6 Bandura’s model of reciprocal determinism, in which observable behavior, environmental events, and internal events are all viewed as interacting with each other. Internal events (person variables) Thoughts and feelings Environmental events Observable behavior 1.2 Five Schools of Behaviorism 31 can be contrasted with the other models we’ve discussed in which environment causes behavior and internal events, if included, serve only a mediational role (Environment ! Internal events ! Behavior). Social learning theory has stimulated a considerable amount of research, particularly in the area of observational learning. It has also stimulated the development of cognitive behavior therapy, in which psychological disorders are treated by altering both environmental variables and cognitive processes. For example, a cognitive behavioral treatment for an irrational fear of spiders might involve some type of safe exposure to spiders (an environmental manipulation) along with instructions to replace fearful thoughts with certain types of calming thoughts (a cognitive manipulation). Cognitive behavioral treatments are quite popular. Social learning theory (along with its cousin, animal cognition) has become a dominant force in behavioral psychology and is rivaled by only one other school of thought—B.!F. Skinner’s radical behaviorism. Yvonne Hemsey/Contributor/Getty Images News/Getty Images theory emphasizes the learning and c importance of o variables. 2. The concept of proposes that ,b , and p variables e and f ) can all influence each other. (including t 3. Bandura’s work has influenced the development of a type of therapy known as – therapy, in which an attempt is made to change behavior by altering both environmental and factors. c Skinner’s Radical Behaviorism From Watson to Bandura, we see a steady increase in the use of internal events to help explain behavior. Not everyone has agreed with this trend. Burrhus Frederic Skinner argued for a return to a somewhat stricter form of behaviorism. Skinner’s version of behaviorism, known as radical behaviorism, emphasizes the influence of the environment on observable (overt) behavior; rejects the use of internal events (including subjectively perceived thoughts and feelings) to explain Burrhus Frederic Skinner behavior; and instead views these thoughts (1904–1990) and feelings as “private” behaviors that need to be explained, just like any other behavior. Thus, unlike Watson’s methodological behaviorism, radical behaviorism does not completely reject the inclusion of thoughts and feelings in a science of behavior, it merely rejects the use of thoughts and feelings as explanations for behavior (Skinner, 1953, 1974). We explain this notion more fully in the following section. Quick Quiz J 1. Bandura’s 32 Chapter 1 Introduction Skinner’s View of Internal Events Skinner viewed internal events, such as sensing, thinking, and feeling, as “covert” or “private” behaviors that are subject to the same laws of learning as “overt” or publicly observable behaviors. Thus, thoughts and feelings can be included in an analysis of behavior, but only as more behavior that needs to be explained. For example, whereas a social learning theorist might say that a student studies because they expect that studying will result in a high mark, Skinner would say that both the act of studying and their expectation about achieving a high mark by studying are the result of some type of experience, such as a history of doing well on exams whenever the student did study. This experience is therefore the cause of both the expectation that studying will result in a high mark and the actual behavior of studying. For several reasons, Skinner was loath to consider internal events as explanations for behavior. First, he agreed with Watson’s concern that since we do not have direct access to the internal events of others, we must rely on their verbal reports of such events, which are often unreliable. Skinner further noted that such unreliability is to be expected, given the manner in which people learn to label their internal events. Specifically, young children need to be taught by their caregivers to describe their internal experiences. Because these caregivers (usually parents) cannot directly observe internal events in their children, they must infer the occurrence of these internal events from the children’s observable behaviors. Consider, for example, the task of teaching a young child to correctly label the feeling of pain. The parent must wait until the child is displaying some observable behavior that typically accompanies pain, such as crying in response to a stubbed toe. Based on this behavior, the parent then infers that the child is experiencing pain and says something like, “Wow, your toe must really hurt!” After a few experiences like this, the child will themself begin using the word hurt to describe what they feel in such circumstances. Pain is probably one of the easier feelings to teach, given that the observable behaviors accompanying it are usually quite distinct (although even here, there may be considerable variability across individuals in the intensity of sensation required before something is called painful). Consider how much more difficult it is to teach a child to accurately describe more subtle emotions, such as contentment or discomfort, for which the observable behaviors are often much less distinct. Because the parents have less reliable information on which to base their inferences about such states, the labels they provide to the child may only approximate the child’s actual feelings. As a result of these experiences, the labels people use to describe their feelings may often be only crude approximations of what they actually feel. For this reason, Skinner was uninterested in using a person’s description of an internal emotional state as an explanation for behavior; he was, however, quite interested in how people come to label their internal emotions (Skinner, 1957). A second problem with using internal events to explain behavior is that it is often difficult to determine the actual relationship of thoughts and feelings to behavior. Take, for example, the act of providing help in an emergency. Do 1.2 Five Schools of Behaviorism you provide help because you feel concern for the person involved? Or do you provide help and feel concerned at the same time, with no necessary link between the two? After all, people often take action in an emergency quite quickly, without reflecting upon how they feel. Or do your feelings of concern for someone sometimes arise after you have tried to help them? Lest this notion seem strange to you, consider that people’s “feelings” can often be altered by manipulating their overt behavior. For example, people can often be induced to change their opinion about a certain issue—such as whether capital punishment should be abolished—if they are asked to write an essay promoting a different point of view. If they do not already hold a strong opinion about that issue and do not feel that they are being forced to write the essay, many people will alter their opinion to be consistent with what they have written (Cialdini, 1993). In similar fashion, the concern you feel for others might sometimes result from, or at least be strengthened by, the act of helping them. A third difficulty with using internal events to explain behavior is that we do not have any means of directly changing these internal events. Our only means of changing both covert behavior (i.e., thoughts and feelings) and overt behavior is to change some aspect of the environment. For example, if I instruct a client to think calm, relaxing thoughts whenever they are in an anxiety-arousing situation, and this effectively reduces the anxiety, a radical behaviorist would say that the effective treatment is not the calm, relaxing thoughts but the instructions I have given the person about thinking calm, relaxing thoughts. And because exposing the client to these instructions is really a manipulation of their environment, then it is really a change in the environment that is ultimately responsible for reducing the level of anxiety. If so, then why not emphasize the environment as the ultimate causeof behavior? A fourth problem with using thoughts and feelings to explain behavior is that (as with explanations based on instinct) such explanations are sometimes only pseudo-explanations. For example, if I say that I “feel like going to the movies,” am I really referring to a bodily condition of some sort? Or am I simply making a prediction about my future behavior? In other words, I am simply saying that I am quite likely to go to the movies under these particular circumstances, given that nothing prevents me from doing so. Thus, my “feeling” statement is really a statement about potential behavior more than about a bodily feeling. For this reason, saying that I am going to the movies because I “feel like going to the movies,” or not studying because I “don’t feel like studying,” is really no explanation at all. For reasons such as these, Skinner (1953) rejected internal events as explanations for behavior; instead, he focused on the environment as the cause of both observable behavior (overt behavior) and internal events (thoughts and feelings). But neither did he believe that we are helpless pawns of our environment. He assumed that once we understand the manner in which the environment affects us, we can then change the environment so that it will exert a more beneficial influence on our behavior, a process that is often referred to as self-control. Although seemingly originating within 33 34 Chapter 1 Introduction ourselves, in!Skinner’s view, even self-control can ultimately be traced to past experiences. Consider, for example, Danielle, who decides to improve her study habits by rearranging her study environment. At first glance, Danielle’s decision (the thoughts she has had about this issue) seems to be the cause of the improvement in her study habits. On the other hand, she would not have made this decision unless she had first been exposed to information about its usefulness. The source of this information (such as reading about such tactics in Chapter 10 and in the appendix on self-management in this textbook) is an environmental influence and is, in Skinner’s view, the ultimate cause of the improvement in her study habits. As for the notion of free will, radical behaviorists are, by definition, interested in the principles by which the environment affects behavior; hence, the notion that a behavior has been freely chosen is not an assumption they would make. They are, however, quite interested in why people might perceive that a behavior has been freely chosen. One possibility is that we perceive our behavior to be freely chosen when the controlling variables are subtle and not easily identified. As previously noted, there is a good deal of evidence suggesting that behavior can be subtly influenced in ways we are not aware of (e.g., Cialdini, 1993). Skinner (1971) also pointed out that we are especially likely to perceive a lack of freedom when behavior is being controlled through the use of aversive consequences. It is for reasons such as this that Skinner strongly argued that society should “engineer” the environment to maximize the use of positive reinforcement (which is perceived as less constraining) and minimize the use of punishment.6 In general, Skinner seems to agree with some aspects of Bandura’s notion of reciprocal determinism, in the sense that environmental events, internal events, and observable behavior can be viewed as interacting with each other. Where Skinner differs, however, is in his assumption that the environment (in combination with genes; next paragraph) ultimately determines both observable behavior and internal events. A diagrammatic depiction of Skinner’s approach might therefore look something like Figure 1.7. (See Skinner [1953, 1987, 1989] for a discussion of his perspective on private events; also see Anderson, Hawkins, Freeman, and Scotti [2000] for the many issues involved in incorporating private events into a science of behavior.)7 6 Note the possible relevance of this process to the rise of extremism in recent years, in which each side tries to threaten and punish those who don’t behave in ways that they deem appropriate. As you will learn in Chapter 9, punishment often creates more problems than it solves, an increase in aggression being one of them. 7 While reading about such issues, bear in mind that behavioristic assumptions are just that—assumptions. They do not necessarily reflect some type of absolute truth, nor do they necessarily reflect the private beliefs of the scientist. Thus, one can adopt these assumptions as a useful way of looking at behavior without abandoning other assumptions, such as a religious belief about the existence of free will. After all, even if free will does exist, the environment still has a major impact on our behavior, and it would be foolish not to learn the principles by which the environment influences behavior. In this regard, the first author recalls a seminary student he once taught who could always be seen carrying around his two favorite textbooks—his behavior analysis text and the Bible. 1.2 Five Schools of Behaviorism 35 Figure 1.7 A diagrammatic representation of Skinner’s view of the relationship between environmental events, internal events, and observable behavior. Although all three components are capable of influencing each other, the emphasis is on environmental events as the ultimate cause of both observable behavior and internal events (as indicated by the solid arrows). Observable behavior Environmental events Internal events (private behaviors) Thoughts and feelings 4. 5. 6. 7. behaviorism. on overt behavior. and f to explain behaviors our behavior and instead regards them as pr influences. that are similarly a function of e In teaching children to label their feelings, parents first have to make an about what the child is feeling based on the child’s inf (overt/internal) behavior. In determining the relationship of thoughts and feelings to overt behavior, it is sometimes difficult to know if the thoughts and feelings pr the behavior. or f Yet another issue with respect to using internal events to explain behavior is that we (can/cannot) directly change such events. Saying that you went skiing because you “felt like it” is, from Skinner’s perspective, explanation for an example of using feelings as a ps your behavior. 8. In Skinner’s view, the act of s -c involves changing some aspect of the environment in order to change our own behavior and is itself influence. ultimately the result of some type of e Skinner’s View of Genetic Factors What about the role of genetic influences on behavior? In discussing these various schools of behaviorism, we have focused on the role of the environment; but we have done so simply because that is what behaviorists generally do—they study the effects of environmental experiences on behavior. Traditionally, they leave it to other disciplines, such as ethology or evolutionary psychology, to study the role of genetic influences. But this does not mean that behaviorists discount the role of heredity. As noted earlier, Darwin’s theory of evolution played a strong role in the establishment of behaviorism, and many behaviorists recognize that heredity can profoundly influence animal and human Quick Quiz K 1. Skinner’s approach to behaviorism is called 2. It emphasizes the effect of the e 3. It also rejects the use of t 36 Chapter 1 Introduction Advice for the Lovelorn While reading this text, you will occasionally encounter advice columns, like this one, in which behavioral concepts are applied to relationship problems. Bear in mind that the advice given is often quite speculative and that real relationship difficulties are too complex to be properly assessed and dealt with through simplistic advice columns. Nevertheless, these columns will, in a fun manner, give you a sense for how behavioral concepts can offer a unique perspective on important aspects of human behavior. Dear Dr. Dee, I have very strong feelings for my new girlfriend, but I can’t tell if they are feelings of infatuation or love. My friends tell me I’m in love, but my parents tell me I’m infatuated. How can I tell the difference? So Confused Dear So, The distinction between love and infatuation is a tough one, and many people find it difficult to differentiate between the two. Interestingly, Skinner (1989) suggested that the more subtle an emotional state, the more value there is in analyzing that emotion in terms of the circumstances that surround it. In what circumstance, for example, are we most likely to use the term infatuation? For starters, are we not more likely to use that term when the level of attachment seems to greatly exceed the rewards available in the relationship? In particular, isn’t it the case that we often apply the word infatuation to a relationship that is driven primarily by desire for short-term sexual rewards? By contrast, the word love is typically applied to a relationship in which a strong level of attachment seems to match the availability of multiple rewards (including, but not limited to, sexual interest!) that are expected to last longer. The relationship seems to have good long-term prospects and is not driven merely by short-term or limited rewards. Thus, for many people, the word infatuation implies an “unhealthy” relationship that is doomed to failure, whereas the word love implies a “healthy” relationship that has the potential to prosper. A little thought will likely reveal other differences between infatuation and love as well as the ways in which people use those labels. Nevertheless, our brief analysis suggests that if you wish to determine whether you are “in love” or “merely infatuated,” you might do well to ponder the rewards offered by that relationship and forget about trying to detect minute differences in feelings. Behaviorally yours, 1.2 Five Schools of Behaviorism behavior. Skinner (e.g., 1953, 1987, 1989), in fact, repeatedly acknowledged that behavior was fundamentally the result of the interaction between genes and the environment. And far from being dismayed by research indicating genetic limitations on operant conditioning (some of which is discussed in Chapter!12), he was fascinated by it and even initiated some early research along these lines (Skinner, 1987). Skinner also noted that operant conditioning bears a striking resemblance to the evolutionary principle of natural selection, with both representing an adaptation to changes in the environment. As earlier discussed, according to the principle of natural selection, members of a species that inherit certain adaptive characteristics are more likely to survive and propagate, thereby passing those characteristics on to their offspring. Thus, over many generations, the frequency of those adaptive characteristics within the population increases and becomes well established. In similar fashion, in operant conditioning, behaviors that lead to reinforcing consequences are more likely to be repeated, whereas those that do not lead to reinforcing consequences are less likely to be repeated. In other words, operant conditioning is sort of a mini-evolution in which an organism’s behaviors that are adaptive (that lead to reinforcers) increase in frequency while behaviors that are nonadaptive (that do not lead to reinforcers) decrease in frequency. The processes of natural selection and operant conditioning are therefore quite similar, with natural selection being concerned with the evolution of inherited characteristics within a species and operant conditioning being concerned with the acquisition of learned behavior patterns within an individual. Although Skinner (1953) was more accepting of the possible effects of heredity on human behavior than was Watson, he nevertheless remained wary about placing too much emphasis on it. Genetic factors are largely unmodifiable within a lifetime, and to assume that a behavior pattern has a strong genetic basis is to assume also that little can be done to alter it (except perhaps through some type of physiological intervention). When dealing with maladaptive characteristics, such as learning difficulties or aggressive tendencies in children, this assumption can have serious implications. Think about it: If you had a son who was having difficulty in math, would you want his teacher to be a strong empiricist (who emphasizes the role of experience in determining behavior) or a strong nativist (who emphasizes the role of genetics and heredity)? Almost certainly, you would want a teacher who is a strong empiricist and who believes the child’s math problems are the result of poor learning experiences, which can therefore potentially be corrected by providing better experiences. Thus, a strong empiricist approach, such as that exemplified by Skinner and other behaviorists, tends to be more optimistic about the possibility of changing behavior for the better. Behaviorists nevertheless have a growing appreciation for the influence of genetic factors on learning and behavior, and recent years have seen a significant increase in research in this area. 37 Quick Quiz L 38 Chapter 1 Introduction 1. Although he emphasized the role of the environment, Skinner also believed that behavior was fundamentally the result of the interaction of g and the environment. He was in fact quite interested in evidence indicating limitations on conditioning. g 2. Skinner believed that the processes of ev and op conditioning were quite similar in that both involved the selection of what was adaptive!from what was not adaptive. 3. On a practical level, Skinner was (enthused/cautious) about genetic explanations for behavior because he believed that such explanations tend to be (optimistic/ pessimistic) about the possibility of change. Behavior Analysis and Applied Behavior Analysis More so than other behaviorists, Skinner was careful to distinguish between the philosophical, scientific, and applied aspects of his approach. Radical behaviorism refers to the philosophical aspect of Skinner’s approach that we have just been discussing, which is the set of assumptions upon which Skinner’s behavioral science (known as behavior analysis) is based. Basic behavior analysis (also called the experimental analysis of behavior) is the basic science that grew out of radical behaviorism and is concerned with investigating the basic principles of behavior, especially the principles of operant conditioning. Such research is often conducted with animals in highly controlled settings. Like Watson, Skinner was concerned that the principles discovered through basic research should have practical application. His work directly led to the establishment of applied behavior analysis, a science and technology of behavior that is concerned with applying the basic principles of behavior toward the analysis and improvement of real-world issues (Cooper, Heron, & Heward, 2020). These applications, which typically use the principles of operant conditioning, range from treating or managing clients with clinical disorders (such as depression and schizophrenia), to improving educational practices and athletic performance, to promoting health-related behaviors and reducing pollution. Applied behavior analysis (often abbreviated as ABA) is particularly well established as the treatment of choice for individuals with developmental and intellectual disabilities, especially autism, and many applied behavior analysts work in this field. Applied behavior analysis is also sometimes called behavior modification or behavior therapy, although the latter term tends to include cognitive behavioral techniques more so than would be found in applied behavior analysis. After taking a course on learning and behavior, some students become interested in pursuing a career in applied behavior analysis. Fortunately, there is a growing demand for individuals with training in applied behavior analysis as well as a growing number of programs available to provide such training. Internationally recognized certification exams have also been established to ensure that applied behavior analysts are highly trained and meet a common set of standards. Individuals with graduate degrees and intensive training in applied behavior analysis are eligible to become 1.2 Five Schools of Behaviorism 39 Board Certified Behavior Analysts (BCBAs). BCBAs conduct behavioral assessments and oversee the delivery of behavioral interventions. Individuals with a bachelor’s degree and training in applied behavior analysis (either during or following their degree) are eligible to become Board Certified Assistant Behavior Analysts (BCaBAs). BCaBAs carry out many of the same functions as BCBAs but do so under the supervision of a BCBA. For detailed information on these accreditations and a list of approved programs for training, go to the Web site of the Behavior Analyst Certification Board at www.bacb.com. For those interested in becoming BCBAs or BCaBAs, this textbook and the course you are presently enrolled in may well be your first step along that path. (See also “The Life of B.!F. Skinner” in the And Furthermore box.) for behaviorism. how best to conduct a science of behavior) is called 2. The basic science that grew out of that philosophy can be called b a (or what is traditionally known as the b a of b ). e 3. The technology that grew out of this basic science is known as (which is often abbreviated ). It is sometimes also called b m . as 4. Individuals with graduate degrees who are highly trained in the technology , are eligible to become board certified while those with bachelor’s degrees are eligible to become board certified . Quick Quiz M 1. Skinner’s philosophy of behaviorism (meaning the set of basic And Furthermore The Life of B.!F. Skinner Though quieter and less colorful than Watson, Skinner was nevertheless also the focus of much controversy. As such, it may be worthwhile to briefly describe his life, especially as he is viewed by many as the prototypical behaviorist. Indeed, in a survey of psychology department chairpersons, Skinner was voted the most influential psychologist of the twentieth century (Haggbloom et al., 2002). Burrhus Frederic Skinner was born in Susquehanna, Pennsylvania, in 1904. Raised in a traditional Presbyterian household, he had a relatively normal childhood, though it was not without difficulties. For example, although he was never physically punished as a child (apart from once having his mouth washed out with soap for saying a bad word), he was taught through reprimands and warnings “to fear God, the police, and what people will think” (Skinner, 1967, p. 407). Interestingly, as a behaviorist, Skinner would later conclude that punishment is an ineffective means for managing behavior, often creating more problems than it solves. 40 Chapter 1 Introduction One of Skinner’s strongest traits, even in childhood, was his love of building and inventing: I made slingshots, bows and arrows, blow guns and water pistols from lengths of bamboo, and from a discarded water boiler a steam cannon with which I could shoot plugs of potato and carrot over the houses of our neighbors. . . . I tried again and again to make a glider in which I might fly. (Skinner, 1967, p. 388) This inventiveness served Skinner well in later years when he was able to build unique devices for studying the behavior of animals, most notably the operant conditioning chamber (or “Skinner box”; see Chapter 6). Without these inventions, it is conceivable that many of the principles discussed in this text would have remained undiscovered. Skinner’s personality was also characterized by a strange mixture of objectivity and sentimentality (Bjork, 1993). For example, when his younger brother Ebbie suddenly died of a cerebral hemorrhage, Skinner observed the death in a surprisingly detached fashion. Nevertheless, he was greatly distressed by the incident and felt pangs of guilt when he later recalled how he had once injured his brother in play. Skinner’s objectivity was also apparent in everyday settings. For example, in describing a family friend, Skinner once wrote: The doctor and his dog are becoming more than idle amusement. . . . . He becomes a fool in the eyes of everyone but me when he attempts to justify the dog’s actions. . . . . [Pep] comes up to us wagging his tail—“He says ‘throw me a stick,’” says the doctor. . . . Lately I’ve got the habit too. It’s quite fun to make up mental processes to fit a dog’s every move. (as quoted in Bjork, 1993, p. 63) As a radical behaviorist, Skinner would later argue that mentalistic terms are often mere inferences derived from observable behavior. Skinner graduated from a small liberal arts college and, with some encouragement from the famous poet Robert Frost, spent a year trying to establish himself as a writer. Although quite disciplined about it, he completed only a few stories and poems and eventually gave up in despair. He sometimes claimed that he had failed as a writer because he had nothing important to say; but he also speculated that his writing was simply too “objective” to interest the average reader, with too few references to thoughts and feelings (Bjork, 1993, p. 56). Years later, Skinner would publish a novel called Walden Two, but it was an “objective” novel about a utopian community founded on behavioral principles. Following his failure at becoming a writer, Skinner came to the conclusion that his real interests lay in the study of behavior. Impressed by the writings of John B. Watson, he entered graduate studies at Harvard in 1928. He thrived in that environment. Much of his graduate and postdoctoral training was surprisingly unstructured, and he was often left to his own devices to study whatever he wished. In later years, he would write that he had no sense of ever devising a theory or testing a hypothesis; he simply followed his interests. He discounted the notion of science as a formal system of theory building and hypothesis testing, asserting that real science is much less structured than most scientists describe it to be (Skinner, 1956). Like Watson before him, Skinner was sometimes the target of false rumors. For example, when Skinner’s wife, Eve, did not adjust well to the “joys” of motherhood, Skinner built an “air crib” (or “baby tender”) to ease the burden of raising their youngest daughter, Deborah. The crib was a large, enclosed space with an unbreakable glass 1.2 Five Schools of Behaviorism 41 Bettmann/CORBIS Figure 1.8 Skinner’s daughter Deborah seemed to thrive in the “air crib” or “baby tender” that her father had built for her, so he set out to market it as an improvement over a standard crib with its jail-like bars and poor hygiene. Unfortunately, it led to the completely false rumor that he was conducting conditioning experiments on his daughters and that they developed severe psychological problems as a result. window. The baby, wearing only a diaper, lay on a woven plastic sheet (the surface of which felt like linen), while the surrounding air was carefully filtered and maintained at a precise temperature (see Figure 1.8). Skinner believed the air crib to be far superior to the jail-like bars, uncertain temperature fluctuations, and loose bedding of the standard crib. It was also much easier to keep clean. Enthused by his invention and by the way Deborah seemed to thrive, Skinner wrote an article on the device for Ladies’ Home Journal and set out to market it. Unfortunately, a story arose that Skinner was isolating his daughter in an “operant conditioning chamber” and experimenting on her. According to one version of the story, the daughter eventually went insane and killed herself. The reality is that she had a happy childhood, spent no more time in the air crib than other children do in a regular crib, and grew up to be quite normal. Nevertheless, the damage was done, and few air cribs were ever sold.8 8 Unfortunately, the myth that Skinner experimented on his daughter still makes the rounds today. A notable example is a book by Laura Slater (2004) entitled Opening Skinner’s Box: Great Psychological Experiments of the Twentieth Century in which she repeats the rumors of Deborah’s insanity. This prompted an angry response from Deborah, entitled “I Was Not a Lab Rat,” in which she demands that people stop spreading these vicious rumors (Skinner-Buzan, 2004). 42 Chapter 1 Introduction In the early 1970s, Skinner was severely criticized by numerous intellectuals and politicians for his book Beyond Freedom and Dignity. In the book, Skinner (1971) rejected the concept of free will and argued that we must instead “engineer” society to more effectively control human behavior. He had hoped the book would encourage people to devise better programs for eliminating pollution, preventing crime, and so on, and he became quite depressed over the criticism he received. Skinner received a more favorable reaction for his invention of the “teaching machine” and programmed instruction—although in later years, he lamented that this notion too had been largely ignored and never used to its full potential. However, the widespread use of personal computers, which are ideally suited for programmed instruction, may eventually change that. Throughout his later years, Skinner remained intellectually active, carefully engineering his environment to compensate for the effects of aging. He even wrote a book, Enjoy Old Age, offering advice on behavior self-management for the elderly (Skinner & Vaughan, 1983). His final public appearance was on August 10, 1990, when he was presented with a Lifetime Contribution Award by the American Psychological Association. Terminally ill, but with little fear of death and as independent as ever, Skinner used his acceptance speech to lambaste the psychological community for its return to mentalistic explanations of behavior. Eight days later, at the age of 86, he passed away from leukemia. (See Vargas, 1990, for a touching description of Skinner’s final days.) If you would like learn more about Skinner’s approach to psychology, his 1953 book, Science and Human Behavior, is considered a classic; however, most students would probably find About Behaviorism (Skinner, 1974) to be more accessible. In Review Instead of summarizing each chapter for you, we will provide you with some suggestions for how to review and reflect upon your own learning. One useful method is for you to create your own summary of the chapter. Write three or four paragraphs that point out key themes in the chapter, identify important researchers or studies, and emphasize any major controversies or unanswered questions. For Chapter 1, you should be able to write one paragraph about what learning is and why it matters to psychologists. A second paragraph should be about important influences on behaviorism, coming from philosophy, early psychology, and Darwinian theory. A third paragraph should identify the five main branches of behaviorism and their founders. Some types of information are best summarized in a table or matrix. For example, you could create a table that lists each of the five branches of behaviorism in one column, the main researcher associated with that branch in the next column, a famous experiment or theory for that branch in the next column, and a sentence that summarizes the perspective in another column. The more columns you add, the greater the detail you can represent in your table. Some students find that this method allows them to visualize differences and organize their ideas. Study Questions You might be thinking to yourself that you’ve already got Quick Quizzes, Study Questions, and a Chapter Test to complete, so what more do we want from you? The way we see it, the more you test your knowledge, the stronger it will become! We also know that the best study tools are the ones you actually use, so if you have limited amounts of time, start with the method that you like best and add others if you can. Alternatively, you might choose to use different methods at different points during the course. Perhaps making your own notes is the best method when you’re studying for weekly quizzes, but using the chapter tests or study questions work best when preparing for a midterm or final exam. You can also try out different forms of media. Writing or typing is not the only way to represent your knowledge. Consider recording yourself as you answer study questions out loud. Describe a graph to someone who can’t see it and have them tell you if it makes sense. Draw a cartoon that represents a disagreement between two famous researchers. Your creativity can help you stay engaged with your education. One thing to note is that the method that is easiest for you is not necessarily the one that will lead to the strongest learning. It’s okay to struggle and make mistakes; in fact, your retention will often be better in the long run if you work a bit harder when learning. You may also need to study differently for different types of tests. Tests of recognition (most multiple choice or true/ false questions) are typically easier than tests of recall (most essay questions), so if you only test yourself with recognition tasks, you may not be well prepared for a final exam that uses recall questions. Ultimately, you decide how you will study. The more that you learn about learning, the better able you will be to make good decisions that enhance your life and help you reach your goals. Further Exploration If you’re interested in how it might feel to be raised for chess mastery, you might enjoy this article about the Polgár sisters and their parents’ behaviorist approaches: www.psychologytoday.com/intl/articles/200507/ the-grandmaster-experiment. If you would like to read some of the historical documents mentioned in this chapter, including original papers and addresses by Watson or Skinner, you can access many of them online at Classics in the History of Psychology, a site maintained by Dr. Christopher Green, York University: https://psychclassics.yorku.ca/. Study Questions Many of the following study questions ask you to compare and contrast different concepts, and, as such, the answers may not always be immediately obvious. As noted earlier, however, having to struggle with the material is often an excellent way to learn it. 43 44 Chapter 1 Introduction 1. Name and briefly describe the two fundamental forms of learning emphasized in this textbook. How do these two forms of learning differ from each other? 2. Describe the nativist versus empiricist approaches to knowledge. How would a nativist versus an empiricist explain how Picasso became such a great artist? 3. Name and provide examples of the four laws of association. Which two laws are most important in modern approaches to learning? 4. Outline Descartes’s dualistic model of human behavior. In his view, what is a basic distinction between the behavior of humans and the behavior of other animals? 5. How did the British empiricists view the acquisition of knowledge? What does the Latin term tabula rasa mean? 6. Describe the structuralist approach to psychology and the basic method by which they gathered data. How did the functionalist approach differ from this? Which of these approaches was in favor of animal experimentation, and why? 7. Describe Darwin’s principle of natural selection. What are the three main components of the principle of natural selection? 8. What is the basic difference between Watson’s methodological behaviorism and Hull’s neobehaviorism? How did Watson’s position on the nature–nurture debate change over time (what was it to begin with, and how did it later change)? 9. How is Bandura’s social learning approach similar to Tolman’s cognitive behaviorism, and how does it differ? 10. How is Skinner’s radical behaviorism similar to Watson’s methodological behaviorism, and how does it differ? 11. What is reciprocal determinism? In what ways would Skinner and Bandura have agreed versus disagreed in their views of how behavior is determined? 12. Outline the distinction between radical behaviorism, basic behavior analysis, and applied behavior analysis. Concept Review applied behavior analysis. A science and technology of behavior that is concerned with applying the basic principles of behavior toward the analysis and improvement of real-world issues. basic behavior analysis. The basic science that grew out of Skinner’s philosophy of radical behaviorism, which experimentally investigates the basic principles of behavior, especially operant conditioning. Also known as the “experimental analysis of behavior.” behavior. Any activity of an organism that can be observed or somehow measured. behaviorism. A natural science approach to psychology that emphasizes the study of environmental influences on observable behavior. Concept Review British empiricism. A philosophical school of thought that maintains that almost all knowledge is a function of experience. cognitive behaviorism. A brand of behaviorism that uses intervening variables, usually in the form of hypothesized cognitive processes, to help explain behavior. Sometimes called “purposive behaviorism.” cognitive map. The mental representation of one’s spatial surroundings. empiricism. In psychology, the assumption that behavior patterns are mostly learned rather than inherited. Also known as the nurture perspective (or, rarely, as nurturism). evolutionary adaptation. A helpful genetic trait (physical or behavioral) that has been shaped through natural selection. functionalism. An approach to psychology that proposes that the mind evolved to help us adapt to the world around us and that the focus of psychology should be the study of those adaptive processes. introspection. The attempt to accurately describe one’s internal conscious thoughts, emotions, and sensory experiences. latent learning. Learning that occurs in the absence of any observable indication of learning and only becomes apparent at a later time. law of contiguity. A law of association in which events that occur in close proximity to each other in time or space are readily associated with each other. law of contrast. A law of association in which events that are opposite from each other are readily associated with each other. law of frequency. A law of association in which the more frequently two items occur together, the more strongly they are associated with each other. law of similarity. A law of association in which events that are similar to each other are readily associated with each other. learning. A relatively enduring change in behavior that results from some type of experience. methodological behaviorism. A brand of behaviorism that asserts that for methodological reasons, psychologists should study environmental influences only on those behaviors that can be directly observed (also known as “classical behaviorism”). mind–body dualism. Descartes’s philosophical assumption that some human behaviors are bodily reflexes that are automatically elicited by external stimulation, while other behaviors are freely chosen. nativism. The assumption that a person’s characteristics are largely inborn. Also known as the nature perspective. natural selection. The evolutionary principle according to which organisms that are better able to adapt to environmental pressures are more likely to reproduce and pass along those adaptive characteristics than those organisms that cannot adapt. neobehaviorism. A brand of behaviorism that utilizes intervening variables, in the form of hypothesized physiological processes, to help explain behavior. (Sometimes called “deductive behaviorism.”) 45 46 Chapter 1 Introduction radical behaviorism. A brand of behaviorism that emphasizes the influence of the environment on overt behavior, rejects the use of thoughts and feelings (internal events) to explain behavior, and instead views thoughts and feelings as private behaviors that themselves can be explained through environmental influences. reciprocal determinism. The assumption that environmental events, observable behavior, and “person variables” (including thoughts and feelings) reciprocally influence each other. social learning theory. A brand of behaviorism that strongly emphasizes the importance of observational learning and cognitive variables in explaining human behavior. Also known as “cognitive social learning theory” or “social cognitive theory.” S–R theory (or stimulus–response theory). The theory that learning involves the establishment of a connection between a specific stimulus (S) and a specific response (R). structuralism. An approach to psychology that assumes that it is possible to determine the structure of the mind by identifying the basic elements that compose it. Chapter Test Chapter tests typically contain fewer hints than quick quizzes do—for example, there is usually only a single blank for an answer, even though the answer may require more than a single word. An answer key has been provided at the end. Note, however, that the question numbers have been scrambled (e.g., the first question on this list is number 9). This allows you to look up the answer to a question immediately without having to worry about inadvertently seeing the answer to the next question. Finally, do not worry if you are unable to answer some of the items without having to refer back to the chapter. Fill-in-the-blank items can be difficult, especially when the information is still relatively unfamiliar to you. You should therefore regard each chapter test as a learning exercise more than a form of assessment. 9. When Tara saw the lush green lawn, it reminded her of just how dry the lawn had been the previous year. Among the four laws of association, this is best described as an example of the law of . 29. Deanna often gets lost when she drives around the city that she lives in. Tolman would say that she has a faulty . 17. When Danielle first saw Gina, she immediately thought of actress Reese Witherspoon. Among the four laws of association, this is most likely an example of the law of . 1. Jordan once became terribly ill while visiting Chicago. As a result, whenever he visits Chicago, he thinks of the illness he suffered at that time. Among the four laws of association, this is best described as an example of the law of . Chapter Test 10. After struggling unsuccessfully to completely eliminate his test anxiety, Andres finally accepts that there are some aspects of himself that he can control and some that he cannot. This conclusion is similar to that of the French philosopher and his theory of dualism. 12. In trying to understand her feelings for Juan, Alisha pays close attention to the sensations she feels each time she sees him. This is an example of the method of . This was a favorite method of research by psychologists who followed the approach known as . 27. Hull’s theory is a (molar/molecular) type of theory, whereas Tolman’s (and Bandura’s) theory is a type. 7. When Anastasia once visited Vancouver, it rained every day for a month. As a result, whenever she is trapped in a rainstorm, it reminds her of her trip to Vancouver. Among the four laws of association, this is best described as an example of the law of . 20. Reggie claims that she freely chose to become a musician, despite her parents putting tremendous pressure upon her to choose a different career. A behaviorist like Skinner would say that her decision to become a musician may instead have been the result of environmental influences that are very (subtle/salient). 15. “My cat never gets lost. It’s like she has a blueprint in her mind of the exact layout of the entire town.” This statement fits best with (name the behaviorist) ’s brand of behaviorism, known as . 11. “Babies know nothing,” Kristie pronounced when her sister commented on how intelligent her new baby seemed to be. Kristie obviously believes that the mind of a newborn is a slate (or, in Latin, ), a notion that was promoted by a group of philosophers known as the . 31. Although Roberta just sits there throughout the lecture, she can afterward repeat everything the professor said. This is an example of learning, which illustrates the distinction between learning and . 16. Skinner noted that people are least likely to perceive their behavior to be freely chosen when their behavior is controlled by consequences. 25. Recall the opening vignette to the chapter where, after making love, one behaviorist comments, “That was fine for you, how was it for me?” This joke is most descriptive of the school of behaviorism. 23. Shira emphasizes environmental explanations for behavior and believes that thoughts and feelings should be regarded as private behaviors that also need to be explained. As such, she is most likely a(n) behaviorist. To the extent that Shira also conducts research into basic 47 48 Chapter 1 Introduction principles of behavior, she can be called a(n) . And to the extent that she applies such principles to developing better methods for coaching basketball, she can be called a(n) . 2. Aristotle was a(n) (nativist/empiricist), whereas Plato was a(n) (nativist/ empiricist). 32. Learning is a relatively (long-term/short-term) change in behavior that results from some type of . 22. When I haven’t eaten for several hours, I feel a strong sense of hunger and therefore walk quickly as I head to the cafeteria. This statement fits best with (name the behaviorist) ’s brand of behaviorism, known as . 5. Neal was recently stung by a wasp and is now quite fearful of wasps. This is best seen as an example of conditioning. 30. John’s therapist tells him that although she cares about what he feels, she is more interested in what he did and in the environmental circumstances that affected both his behavior and his feelings. This therapist’s approach fits best with ’s brand of behaviorism, known as . 19. Descartes believed that the behavior of is entirely reflexive. 14. Mandy found a five-dollar bill when she took out the trash one day. As a result, she now volunteers quite often to take out the trash. This is an example of conditioning. 26. A middleman in a business transaction is analogous to what Tolman and Hull referred to as a(n) variable. 33. As originally defined by Watson, behaviorism is a(n) science approach to psychology that emphasizes the study of influences on behavior. 3. After Jasmine saw her sister talk back to the sassy kid next door, she talked back too. The behaviorist who would find this incident most interesting is probably . 36. Learning how to swing a bat by watching others is an example of (observable/observational) learning. Actually swinging the bat while other people are watching is an example of (observable/observational) behavior. 18. Angie’s therapist tells her that he doesn’t care what she thinks and feels; he is concerned only about what she did and about the circumstances that affected her behavior. This therapist’s approach fits best with (name the behaviorist) ’s brand of behaviorism, known as . 8. In considering the process of dreaming, a researcher who followed the approach to psychology known as would have been most concerned with understanding how dreaming facilitates our ability to adapt to the world around us. Answers to Chapter Test 35. Lynne persists in teaching her daughter music despite the insistence of her husband that the child “was born tone deaf.” Which of these two has an attitude most similar to that of a behaviorist? 24. Sal claims that the neglect he suffered as a child resulted in low selfesteem, which in turn resulted in his long history of criminal activity. His parole officer tells him that such an explanation is too simplistic, that it ignores the complex manner in which the various facets of life interact with each other, and that Sal needs to acknowledge that his own attitude played a role in creating his difficulties. Among the theorists in this chapter, the one who would most appreciate this statement is , because it agrees with his concept of determinism. 4. “Great musicians are born, not made” is an example of the (nativist/ empiricist) perspective on behavior, and “practice makes perfect” is an example of the (nativist/empiricist) perspective. 28. (Hull/Watson/Both) assumed that behavior consists of a long chain of specific stimulus–response connections. This approach is known as a(n) theory of behavior. 13. William James was a (structuralist/functionalist), and Edward Titchener was a (structuralist/functionalist). 6. Methodological behaviorism differs from radical behaviorism in the former’s (rejection/acceptance) of thoughts and feelings in the analysis of behavior. 21. Removing the television set from your room so you won’t be distracted while studying is an example of Skinner’s approach to , which emphasizes (using willpower/changing the environment) to alter our behavior. 34. The Rs in the 3R approach to studying stand for , , and . Answers to Chapter Test 1. contiguity 2. empiricist; nativist 3. Bandura 4. nativist; empiricist 5. classical 6. rejection 7. frequency 8. functionalism 9. contrast 10. Descartes; mind–body 11. blank; tabula rasa; British empiricists 12. introspection; structuralism 13. functionalist; structuralist 14. operant 15. Tolman’s; cognitive (or purposive) behaviorism 16. aversive (or punishing) 17. similarity 18. Watson’s; methodological behaviorism 19. animals 20. subtle 21. self-control; changing the environment 49 50 Chapter 1 Introduction 22. Hull’s; neobehaviorism 23. radical; basic behavior analyst; applied behavior analyst 24. Bandura; reciprocal 25. methodological behaviorism 26. intervening 27. molecular; molar 28. both; stimulus–response (S–R) 29. cognitive map 30. Skinner’s; radical behaviorism 31. latent; performance 32. long-term; experience 33. natural; environmental; observable 34. read, recite, review 35. Lynne 36. observational; observable Chapter 2 Research Methods The following is based on an actual conversation that took place between a “relationship expert” and a caller on a radio call-in show: “Hi, Dr. Kramer. I need some advice. I’m wondering if I should get married or break off my engagement and finish university first.” “How old are you?” “Twenty-one.” “Break off your engagement. Statistically, your marriage has a much better chance of surviving if you don’t get married until your late twenties.” “Oh, okay.” Chapter Outline 2.1 Basic Terms and Definitions Independent and Dependent Variables Functional Relationships Stimulus and Response Overt and Covert Behavior Appetitive and Aversive Stimuli Motivating Operations Contiguity and Contingency 2.2 Measurement of Behavior Behavioral Definitions Recording Methods Assessing Reliability 2.3 Research Designs Descriptive Research Experimental Research 2.4 Use of Animals in Behavioral Research Learning Objectives 2-1 Use accurate terminology for describing research and the measurement of behavior. 2-2 Select appropriate methods for measuring various types of behavior. 2-3 Distinguish between descriptive and experimental research methods. 2-4 Select appropriate research designs for various research questions or situations. 2-5 Outline the pros and cons of using animals in behavioral research. 51 52 Chapter 2 Research Methods This chapter introduces you to the basic methods of behavioral research. Once a researcher has developed a hypothesis or has decided on a specific topic of interest, such as the effect of reward size on speed of learning, they will employ a research method to obtain some behavioral data. Some of the methods for obtaining data include naturalistic observation, control group designs, and single-subject designs. The methods used in behavioral research are in many ways similar to those used in other fields of psychology. For example, behavioral research often involves comparisons between “experimental” groups that receive some kind of manipulation (or treatment) and “control” groups that do not receive that manipulation. In some cases, however, the methods are quite distinctive. For example, behavior analysts (as discussed in Chapter 1, these are behaviorists who follow Skinner’s philosophy of radical behaviorism) have a strong preference for conducting experiments that require only one or, at most, a few subjects. These types of experimental designs, known as single-subject designs, have several advantages (as well as disadvantages), which we discuss later in the chapter. Let’s begin, however, with an overview of some basic terms and definitions. 2.1 Basic Terms and Definitions Independent and Dependent Variables All scientific research involves the manipulation and/or measurement of certain variables. A variable is a characteristic of a person, place, or thing that can change (vary) over time or from one situation to another. Temperature is an example of a variable: temperature varies from day to day, season to season, and place to place. Height and weight are also examples of variables—people come in many different sizes and shapes. Until a person reaches maturity, their height will change over a period of time. Weight is even less consistent and can fluctuate endlessly, often in directions we do not particularly like. Almost anything can be considered a variable. Consider this ad: Puppies for sale! Friendly male and female Standard Poodle pups are 10 weeks old and ready for their forever homes. Colors include white, brown, and apricot. All puppies are well socialized in a home with small children, vaccinated, and guaranteed to be adorable! There are several types of variables in this ad. “Friendly” is a variable that refers to a range of temperament or personality; that variable might range from friendly to shy. This sort of variable is measured on a spectrum or continuum, from low to high levels of the trait. Color could be considered a categorical variable (white, brown, apricot) but could also be measured on a spectrum (light brown to dark brown). The sex of puppies is another categorical variable, where “male” and “female” are most common, but intersex categories also exist. (For humans, we might also talk about gender, which is distinct from sex and is increasingly understood to be on a spectrum.) 2.1 Basic Terms and Definitions There are other examples of variables in this ad, including level of socialization, vaccination status, and age. Adorableness is, of course, an important variable for a puppy! Two types of variables are particularly important in an experiment. The independent variable is the aspect of an experiment that is made to systematically vary across the different conditions in the experiment. In other words, the independent variable is what is manipulated in an experiment. For example, we may be interested in whether the size of a reward (or “reinforcer”) can affect the efficiency of learning. To test this notion, we might conduct a maze learning experiment with rats. Each rat is given 10 trials in which it is placed in a maze and allowed to find its way to the goal box. Depending on the “experimental condition” to which the rat has been randomly assigned, it receives one, two, or three pellets of food each time it reaches the goal box. Thus, the independent variable in this experiment is the number of food pellets the rats in each group receive when they reach the goal box. The dependent variable is the aspect of an experiment that is allowed to vary freely to see if it is affected by changes in the independent variable. In other words, the dependent variable is what we measure in an experiment. In a psychology experiment, this is almost always some type of behavior. Changes in the dependent variable are dependent upon changes in the independent variable (which is a useful phrase to remember to help you distinguish between dependent and independent variables). In our rat experiment, the dependent variable could be the total number of errors (i.e., number of wrong turns) the rat makes while trying to find its way to the goal box. Alternatively, we might simply look at the speed with which the rat reaches the goal box. Either way, a significant difference between groups on this measure will indicate whether the number of food pellets found in the goal box affects the rat’s efficiency in learning the maze. In turn, this will provide supportive evidence for our more general notion—which is what we are really interested in—that the size of a reinforcer affects the efficiency of learning. Another type of variable that closely resembles an independent variable, and is typically labeled as such, is a quasi-independent variable (“quasi” means resembling). A quasi-independent variable is one that is inseparable from an individual and cannot be manipulated; however, group comparisons can be made based on that variable. Designs that use such variables are called quasi-experimental designs. For example, nationality is not something to which people can randomly be assigned, but researchers use nationality as a quasi-independent variable when they compare the attitudes of Australian, Chinese, and Sudanese students. There are a number of variables like this that cannot be experimentally manipulated, including age, sex or gender, species, and other physical characteristics. When you come across examples of research that includes such variables—such as comparative designs that examine differences in learning ability between species—don’t be confused by the fact that the independent variable isn’t actually being manipulated by the researcher. It still functions like an independent variable in that the researcher is looking to see if it is associated with differences in a dependent 53 54 Chapter 2 Research Methods variable. Thus,!in the preceding example, nationality can be regarded as an independent variable that is being used to examine differences in the dependent variable of student attitudes. Functional Relationships Quick Quiz A In behavioral research, the dependent variable is almost always some behavior, and the independent variable is some environmental event or experience that is presumed to influence the behavior. The relationship between changes in an independent variable and changes in a dependent variable is known as a functional relationship. Thus, behaviorists are typically interested in discovering functional relationships between changes in environmental events and changes in behavior. A functional relationship can also be thought of as a cause-and-effect relationship, with changes in the independent variable being the cause and changes in the dependent variable being the effect. 1. A researcher is interested in studying the effects of viewing television violence on aggression in children. She shows one group of participants an extremely violent movie, another group a moderately violent movie, and a third group a nonviolent movie. In this case, the level of movie violence shown to the children would be variable, and the children’s subsequent considered the variable. level of aggressive behavior would be the 2. A change in the dependent variable is considered to be the (cause/effect) in an experiment, whereas a change in the independent variable is considered to be the . 3. A relationship is the relationship between a change in an independent variable and an associated change in a dependent variable. Behaviorists are typically concerned with discovering such relationships events and changes in between changes in e . b 4. A variable like age, nationality, or gender is not manipulated but can be treated like variable for the purpose of making comparisons an i between groups. This type of variable is known as a q -independent variable. Stimulus and Response Two terms commonly encountered in behavioral textbooks are stimulus and response. A stimulus is any event that can potentially influence behavior. A response is another word for behavior, but one that tends to be used when discussing a particular instance of a behavior, especially when it occurs in reaction to a stimulus. For example, a loud noise is a stimulus that elicits a startle response. Similarly, loud music (a stimulus) might cause your neighbor to bang on the wall (a response), and a high mark on a test (a stimulus) might cause you to grin with delight (a!response). Although it would not be incorrect to use the word behavior rather than response in each of these examples, “response” is preferable because the examples refer to specific instances of those behaviors. 2.1 Basic Terms and Definitions The plural for the word stimulus is stimuli. Thus, a red light is a stimulus, and a red light and a green light are stimuli. A stimulus is sometimes also referred to as a cue in that it signals (cues) the occurrence of a certain behavior, such as when a red traffic light serves as a cue for stopping and a green traffic light serves as a cue for proceeding. Note that the response of one organism can act as a stimulus that influences the response of another organism. For example, when one rat bites another, the bite is a stimulus that might elicit a retaliatory response from the other rat. In turn, this retaliatory response might then act as a stimulus that induces the first rat to retreat. Likewise, a smile from Shane is a stimulus that encourages Tami to say hello; Tami’s hello is in turn a stimulus that encourages Shane to introduce himself. Thus, social interactions generally consist of a chain of alternating responses, with each person’s response acting as a stimulus for the next response from the other person. Overt and Covert Behavior It is also important to distinguish between overt and covert behavior. Overt behavior is behavior that can potentially be observed by an individual other than the one performing the behavior. In other words, it is behavior that could be publicly observed if others were present. A person’s response of saying hello and a rat’s response of pressing a lever are both instances of overt behavior. As noted in Chapter 1, behaviorists traditionally have emphasized the study of overt (observable) behavior. Skinner, however, maintained that internal events such as thoughts, feelings, and even sensory experiences (e.g., seeing and hearing) should also be classified as behaviors. Skinner referred to such behaviors as “private behaviors” or “private events,” although they are more commonly referred to as covert behaviors. Thus, covert behavior is behavior that can be perceived only by the person performing the behavior. In other words, it is behavior that is subjectively perceived and not publicly observable. Dreaming, thinking about your next chess move, visualizing how your date will go on the weekend, and feeling anxious are all examples of covert behavior. Of course, some covert behaviors have components that could be made publicly observable. A feeling of anxiety, for example, is likely to involve increases in heart rate and muscle tension, both of which could be electronically measured. If such measurements are made, then those particular components of anxiety can be considered overt—which, from a behavioral perspective, is much preferred over a purely subjective report of anxiety. Just as the behavior of one person can serve as a stimulus for the behavior of another, covert and overt behaviors within the same person can act as stimuli for each other. For example, thinking about one’s next move in chess (a covert behavior) is a stimulus that influences which chess piece you actually move (an overt behavior), while accidentally moving the wrong chess piece (an overt behavior) is a stimulus that induces you to think unpleasant thoughts about yourself (a covert behavior). As behavior analysts put it, the environment does 55 Chapter 2 Research Methods © 1985 universal press syndicate 56 not stop with the skin: events both outside the skin and inside the skin can influence our behavior—though behavior analysts maintain that the ultimate cause of the behavior is often to be found outside the skin. (For example, what might a behavior analyst consider to be the ultimate cause of a person thinking about a certain chess move and then making that move?) Appetitive and Aversive Stimuli Quick Quiz B Many stimuli, both internal and external, can be classified as appetitive or aversive. An appetitive stimulus is an event that an organism will seek out. Food is an appetitive stimulus when we are hungry; water is an appetitive stimulus when we are thirsty. An aversive stimulus is an event that an organism will avoid. Electric shock and extreme heat are examples of aversive stimuli. (Note that the word is aversive and not adversive.) Appetitive and aversive stimuli could also be defined as those events that people usually describe as pleasant or unpleasant. Such descriptions are often accurate, but one has to be careful not to rely on them too much. People can vary widely in the types of events they regard as appetitive versus aversive—a point that many parents overlook when they attempt to reinforce or punish a child’s behavior. As well, a person may claim that a certain experience is unpleasant, yet work actively to obtain it. For example, someone might describe their pack-a-day smoking habit as “disgusting,” yet will do everything possible to make it to the store in time to buy another pack. Despite what they say, tobacco is clearly an appetitive stimulus for them. The moral of the story is that what we say is often unreliable—or, as behavior analysts sometimes put it, “just verbal behavior.” It may or may not accurately reflect the nonverbal behavior it presumably describes. 1. A(n) is any event that can potentially influence is a term that is often used when behavior; a(n) referring to a specific instance of a behavior. 2. A tone is a s , and a tone and a bell are s . 3. In a social interaction, one person’s response can be another person’s . 4. Julie dislikes Jake, one of the people who work in her department. Because Julie avoids Jake as much as possible, Jake can be considered an (appetitive/adversive/ aversive) stimulus. 2.1 Basic Terms and Definitions 5. Julie closes her office door when Jake is nearby, which is an example of a(n) (overt/ covert) behavior. Julie also thinks unkind thoughts about Jake and feels angry when he is around, both of which are examples of behavior. 6. Russ loves dogs, and often goes to the local dog park to eat his lunch so that he can watch the dogs playing. Based on Russ’s behavior, dogs are stimuli for him. 7. If we think before we act, then our (covert/overt) behavior serves as a stimulus behavior. If we act first and feel regret that influences our behavior serves as a stimulus that influences later, then our behavior. our 8. Covert behaviors can also be called p behaviors or events. Motivating Operations You may have noticed in some of the preceding examples that how we react to an event depends on a particular state or condition. Any procedure that affects the appetitiveness or aversiveness of an event is called a motivating operation. There are two types of motivating operations: establishing operations and abolishing operations. An establishing operation is a procedure that increases the appetitiveness or aversiveness of an event, and an abolishing operation is a procedure that decreases the appetitiveness or aversiveness of an event. For example, depriving an animal of food is an establishing operation that increases the appetitive value of food, while “satiating” an animal on food—that is, feeding it until it is no longer hungry—is an abolishing operation that decreases the appetitive value of food. Thus, deprivation (the prolonged absence of an event) and satiation (the prolonged exposure to or consumption of an event) are common types of motivating operations. If the event is being used as a reinforcer for some behavior—such as food being used as a reinforcer for lever pressing—then we could also regard deprivation and satiation as procedures that respectively increase or decrease the effectiveness of a reinforcer. The manner in which deprivation and satiation affect the appetitiveness of food is obvious. Less obviously, deprivation and satiation can also affect our reactions to many other events. If you have ever gone without television for an extended period of time (deprivation), you may have found it surprisingly interesting when you finally saw it again. Likewise, if you hear a favorite song too often (satiation), you may lose interest in it. Although the general rule is that deprivation increases the appetitiveness of an event and satiation decreases its appetitiveness, exceptions can occur. For example, people (and even rats, as you will read in a later chapter) who undertake severe diets sometimes acquire a disorder known as anorexia nervosa. In these cases, severe food deprivation seems to decrease the 57 58 Chapter 2 Research Methods appetitive value of food rather than increase it, and these individuals begin to eat even less food than the diet allows. They may even begin to actively avoid food, in which case food deprivation can paradoxically become an establishing operation for the aversiveness of food! Many people who have anorexia also engage in extremely high levels of activity yet seem to find the activity more, not less, reinforcing—that is, they do not seem to “satiate” on the activity. These processes are discussed more fully in Chapter 12.1 Contiguity and Contingency Two terms that are often confused are contiguity and contingency. Although they sound similar, they have very different meanings. Contiguity, as mentioned in the opening chapter, means “closeness or nearness.” Thus, temporal contiguity is the extent to which events occur close together in time. Thunder and lightning are temporally contiguous—we see the lightning and shortly afterward hear the thunder. Temporal contiguity is an important aspect of learning. A rat will more readily learn to press a lever for food if the food immediately follows the lever press than if it appears several seconds later. Likewise, a child will more quickly learn to throw a tantrum to get access to an iPad if they are immediately given the iPad each time they do so. Spatial contiguity is the extent to which events are situated close to each other in space. This type of contiguity is also an important aspect of learning. It is easier for a rat to learn to press a lever for food if the food dispenser is close to the lever as opposed to being several feet away. Likewise, it may take a young child (or a young puppy) somewhat longer to learn that a doorbell, as opposed to a knock, indicates that someone is at the front door. The sound of the knock is spatially contiguous with the door (the sound comes from the door), whereas the sound of the doorbell is not (the sound usually comes from a box located elsewhere in the house). The term contingency has a quite different meaning from contiguity. A contingency is a predictive (or functional) relationship between two events, such that the occurrence of one event predicts the probable occurrence of another. If a rat receives a food pellet whenever it presses a lever, then a contingency exists between lever pressing and food. We then say that the presentation of food is contingent on lever pressing. Likewise, if a child receives a big balloon each time they go to the dentist, then a contingency exists between visiting the dentist and receiving the balloon. In other words, receiving the balloon is contingent upon visiting the dentist. As you will see later, contingency is an extremely important aspect of learning. 1 If you are uncertain how to pronounce satiation, or any other word in this text, audio examples of pronunciation can be found on the Internet, such as at the Merriam-Webster site: www.merriam-webster.com/help/audiofaq.htm. 2.2 Measurement of Behavior 2. 3. 4. 5. 6. 7. 8. o is a procedure that affects the appetitiveness or aversiveness of a stimulus. An a o is a procedure t h a t m a k e s a n e v e n t l e s s a p p e t i t i v e o r l e s s a v e rs i v e , w h i l e a n o makes an event more e appetitive or more aversive. Farah has been working out of town and has not seen a movie for over a year. It is likely that, for her, the reward value of going to a movie has (increased/ decreased) as a function of (satiation/deprivation), which is an example of a(n) operation. The term means “closeness or nearness.” Erin claims that she once experienced a sharp pain in her leg at the precise moment that her son, who was on a mountain-climbing expedition, broke his c between her leg. Because of the t feeling of pain and her son’s injury, Erin now believes that she has some type of psychic ability. People who live close to each other are more likely to date and fall in love. In c seems to other words, s have a strong effect on the development of romantic relationships. Sasha obtains a high mark on her exams only when she studies diligently, even if the exam was delayed for several days. For Sasha, there is a (contingency/ contiguity) between studying diligently and doing well on her exams. The fact that a dog receives a treat only when it begs means that the treat is (contiguous with/contingent upon) the behavior of begging. The fact that the treat closely follows begging also means that the two events are (contiguous with/contingent upon) each other. 2.2 Measurement of Behavior Behavioral Definitions When we study the effects of particular variables on a certain behavior (which is often referred to as the “target behavior”), it is important that we properly define the behavior. Such behavioral definitions should be objective in the sense that they refer to some observable aspect of the individual’s behavior. For example, yelling and striking are observable aspects of aggressive behavior, but feelings of anger are not. Therefore, defining aggression in terms of the physical characteristics of yelling and striking is more precise than defining it as feelings of anger. Behavioral definitions should also be clearly defined, that is, unambiguous. For example, we might define yelling as a loud vocalization that continues for more than five seconds and can be heard outside a closed door. Striking might be defined as a rapid arm or leg movement that results in physical contact. From a scientific perspective, an unambiguous definition will ensure that our measurements of the behavior are relatively consistent over time and across settings. What counts as an aggressive incident today will also count Quick Quiz C 1. A m 59 60 Chapter 2 Research Methods as an aggressive incident tomorrow. Further, if we are investigating various treatments to reduce the number of aggressive incidents (e.g., by rewarding the child for acting nonaggressively), we can be more certain that any observed change in the aggressive behavior is the result of our treatment as opposed to an unconscious shift in our definition of aggression. Finally, an unambiguous definition will make it easier for other researchers to replicate our results. Clear definitions of behavior are also beneficial outside the clinical or research setting, particularly in such tasks as child rearing (the ultimate challenge in behavior management). A major problem faced by many children is that parents often shift their standards as to what constitutes appropriate behavior. For example, a parent might constantly tell a child that eating in the living room is wrong, but then allow eating in the living room when visitors arrive or when the family is watching a movie. A clearer definition of what behaviors are appropriate versus inappropriate would be less confusing to the child and would reduce the probability of the child violating the rules. One parent we know of uses a “three-warning” rule for situations that require compliance but not necessarily immediate compliance. For example, if one of the children is asked to get ready to go swimming with their aunt, they must comply by the third warning or else suffer a negative consequence (e.g., they will not be allowed to go swimming that day). Because the rule is so well defined and allows the child a certain amount of time “to get mobilized,” negative consequences rarely have to be imposed. And even when the child does not comply and does suffer the consequences, they rarely make a fuss about it because they are well aware of the contingencies from the outset. (This does not mean that the children in this family are rigidly controlled. In fact, one’s first impression upon entering the household is that it is quite chaotic, with children running everywhere, laughing and playing. Within clearly defined limits, the children are allowed a great deal of freedom, which they very much appreciate.) Recording Methods Depending on how we define a target behavior, there are several ways in which we can go about measuring it. Let’s look at a few of these methods. Rate of Response One of the most popular measures in behavioral research is rate of response, which is the number of responses that occur in a certain period of time (i.e., the frequency of the response). Rate measurements are most appropriate when each response is very brief, with a well-defined start and finish (i.e., onset and offset). The number of cigarettes smoked per day, the number of words written in a one-hour writing session, and the number of body slams in a half-hour broadcast of professional wrestling are all rate measures of behavior. (When defining or giving examples of rate, be sure to include the time aspect. For example, number of lever presses per hour is a rate measure of behavior, but number of lever presses by itself, with no mention of time, is not.) 2.2 Measurement of Behavior Certain experimental procedures have been explicitly designed to facilitate measuring behavior in terms of rate. For example, operant conditioning experiments often involve rats pressing levers to earn food. The lever press is a very definable response, because once the lever is pressed sufficiently for the microswitch to be activated, a response is electronically recorded. Number of lever presses per session thus provides a precise measure of the rat’s food-directed behavior. Rate is a particularly favored measure among some behaviorists (especially radical behaviorists), because it tends to be highly sensitive to the influence of other variables. For example, the rate at which a rat presses a lever for food will vary closely with the number of hours of food deprivation, the type of food being delivered (preferred or non-preferred), and the amount of effort required to press the lever. Likewise, the rate at which you can say aloud the definitions of the concepts in this text (e.g., the number of terms you are able to define per minute while sorting through your flash cards) may be a good indicator of the extent to which you are now fluent enough with a concept to begin applying it to different situations. In fact, such fluency measures of learning are a basic aspect of a behavioral approach to instruction known as precision learning (e.g., Lindsley, 1991; Binder, 1996). A cumulative recorder is a classic device that measures the total number of responses over time and provides a graphic depiction of the rate of behavior. This instrument consists of a roll of paper that unravels at a slow, constant pace and a movable pen that makes tracks across it (see Figure!2.1). If there are no responses for a period of time, the pen remains stationary while the paper unrolls beneath it. This results in a flat, horizontal line along the paper, with!longer lines indicating longer periods of no responding. When a!response occurs (e.g., the rat presses the lever), electronic equipment registers the response and produces a slight upward movement of the pen. Figure 2.1 Illustration of a cumulative recorder. This device consists of a roll of paper that unravels at a slow, constant pace. If no response is made, the pen remains stationary, resulting in a horizontal line. A high rate of response produces a steep line, and a low rate of response produces a shallow line. The short slashes indicate the points at which reinforcers were delivered; for example, food pellets delivered to a rat for making a certain number of lever presses. (Source: Malone, 1990.) Paper movement Pen stepped by responses Rapid responding Slow responding No responding Reinforcements Roller 61 62 Chapter 2 Research Methods Thus, a!low rate of response produces a line that slopes upward at a shallow angle (because the pen is slowly moving upward while the paper passes beneath it), whereas a high rate of response produces a line that slopes upward at a steep angle. The important thing to remember is that the steeper the line, the higher the rate of response. A cumulative record thereby provides an easily read, graphic depiction of changes in the subject’s rate of response over time. (Needless to say, these days, response rates are more often recorded by computer software programs. This allows for the generation of various types of descriptive records, including cumulative records.) Intensity The intensity of a behavior is the force or magnitude of the behavior. For example, in Pavlov’s classical conditioning procedure with dogs, in which a tone was followed by food, Tone: Food → Salivation Tone → Salivation the strength of conditioning was typically measured as the amount (magnitude) of saliva produced whenever the tone was presented by itself. More saliva indicated stronger conditioning. Another example of intensity is the force with which a rat presses a lever to obtain food. Likewise, it is intensity that we are concerned with when we teach a child to speak softly and print firmly. Duration Duration is the amount of time that an individual repeatedly or continuously performs a behavior, with no concern for how fast or slow the behavior is occurring. This measure is appropriate when we are concerned with either increasing or decreasing the amount of time the behavior occurs. For example, a student may attempt to increase the amount of time spent studying each week, as well as decrease the amount of time spent watching television. Speed Although duration measures are sometimes useful, they are in some ways imprecise. You may run for an hour, but the speed at which you run (as indicated by the amount of distance covered during that hour) will be a more accurate indicator of your fitness level. Thus, speed is the length of time required to perform a behavior from start to finish, which indicates how fast or slow the behavior is occurring. The length of time it takes for a rat to run through a maze from the start box to the goal box is a measure of speed. (See Figure 2.2 for examples of mazes that have been used in behavioral research.) We are also concerned with speed when we teach a child to eat a meal more quickly (if he tends to dawdle at the dinner table) or more slowly (if he tends to fling food everywhere in a rush to get finished). Studies on activity levels in rats often measure both the duration of running (the amount of time the rats spend running in the wheel) as well as the speed of running (the distance covered per hour or per day). But note that we could also use a rate measure of wheel running, that is, the number of wheel turns per hour. 2.2 Measurement of Behavior Figure 2.2 Three classic mazes used in behavioral research. Although the Hampton Court maze was often used by researchers in the early 1900s, it was later largely supplanted by the T-maze and the straight-alley “maze,” which, because of their simplicity, proved more useful for investigating basic principles of behavior. (Source: Lieberman, 2000.) Goal box Goal box Food Food Start box (b) T-maze (a) Hampton Court maze Food Start box Goal box (c) Straight-alley maze Latency The latency of a behavior is the length of time required for a behavior to begin. With respect to conditioning of salivation, the strength of conditioning can be measured not only in terms of the amount of saliva but also in terms of how soon the dog begins salivating after it hears the tone. Likewise, the number of days it takes for a student to begin working on a term paper after it has been assigned might be a useful measure of the extent to which they procrastinate on such tasks. TV game shows that require contestants to press buzzers when they believe they have the right answer are using a latency measure of the contestants’ performance. Latency, speed, and duration are often confused because they all involve some type of time measurement. To help distinguish between them, consider the behavior of an athlete who specializes in the 100-meter sprint. The amount of time it takes to commence running after hearing the starting pistol—which is only a fraction of a second—is a measure of latency, whereas the amount of time it takes to complete the race (start to finish) is a measure of speed. Conversely, the total amount of time spent running each day (independent of how fast they are running) is a measure of duration. Interval Recording A particularly efficient way of measuring behavior, often utilized in applied settings, is interval recording: the measurement of 63 64 Chapter 2 Research Methods whether a behavior occurs during each interval within a series of continuous intervals. For example, if we wish to measure the amount of aggressive behavior in a classroom, we might make a video record of several hours of class time. We would then have observers view the video and note whether an aggressive incident occurred within each successive 10-minute interval. The proportion of intervals in which at least one incident occurred would then be our overall measure of aggression. For instance, imagine that we record three hours (180 minutes) of class time. We then have observers view the video and count the number of 10-minute intervals in which least one aggressive incident occurred. Note that we are not concerned with how many aggressive incidents occurred in each interval, only with the number of intervals in which at least one aggressive incident occurred. The percentage of intervals during which at least one incident occurred is then our overall measure of the behavior. For example, if at least one aggressive incident occurred in 12 of the 18 ten-minute intervals, then the overall measure of aggression would be 12/18 " 100 = 66.7%. A major advantage of interval recording is that one does not have to record every single response, which may be difficult if responses occur at a high rate or if it is difficult to count separate responses (e.g., an argument that consists of a rapid exchange of aggressive vocalizations). Interval recording is also useful if it is difficult to determine the point at which the target behavior starts and stops. Arguments would be a good example of this in that they sometimes build slowly and can be punctuated by periods of silence; trying to determine the exact moment when the argument begins and ends may be difficult. Time-Sample Recording A variant of interval recording is time-sample recording. In time-sample recording, one measures whether a behavior occurs during each interval within a series of discontinuous intervals (intervals that are spaced apart). For example, to assess the level of aggression in a classroom, we might have an observer unobtrusively enter the classroom for a 10-minute interval at the start of each half hour and record whether at least one aggressive incident occurred during that interval. The behavior of the students is thus intermittently sampled, and the percentage of these sampled intervals in which an aggressive incident occurred is our overall measure of aggression. Imagine, for example, that over a period of six hours, we are able to sample 12 ten-minute intervals. If one or more aggressive incidents occurred in nine of those intervals, then the overall level of aggression is calculated as 9/12 " 100 = 75%. Although we will not have observed every act of aggression using such a method, we will nevertheless have obtained a fairly good approximation of the level of aggression in that setting. As well, this method of recording is very time efficient for our observer, who can spend most of the day working on other tasks or making observations in other classrooms. Topography Sometimes we are concerned with the topography of a behavior, which is the physical form of the behavior. For example, rather than 2.2 Measurement of Behavior record the rate at which a rat presses a lever, we might observe how it presses the lever, such as whether it uses its left paw or right paw. Similarly, it is the topography of the behavior that we are concerned with when we teach a child how to dress appropriately, write neatly, and brush their teeth properly. Training a dolphin (or your pet goldfish) to swim through a hoop to obtain a food reward is yet another example in which the topography of the behavior is the focus of concern. Number of Errors Any behavior in which responses can be categorized as right or wrong can be assessed in terms of the number of errors. For example, the number of wrong turns a rat takes before it finds its way through a maze to the goal box is one measure of how well the rat has learned the maze. Likewise, the number of errors a student makes on an exam is the traditional method for determining how well the student knows the material. Assessing Reliability One thing you may have wondered about in reading some of the preceding examples is the reliability or trustworthiness of the data that we are gathering using these methods. This is an important issue when the data is being gathered by observers who might vary widely in their judgments as to whether a particular behavior has occurred. For this reason, where possible, researchers attempt to measure interobserver reliability (also known as interobserver agreement), which is the extent to which two or more independent observers agree in their observations of the occurrence of a behavior. For example, in an interval recording procedure in which two observers independently record the occurrence of aggression in each of 12 consecutive intervals, they may agree on whether an incident occurred in 10 of the intervals and disagree in 2 of the intervals (see Table 2.1). Our measure of interobserver reliability in this case can be calculated as the number of intervals during which the observers agree divided by the total number of intervals: 10/12 = 83.3%. There are no hard and fast rules for what constitutes adequate reliability, but 80% is often regarded as the minimum acceptable level and 90% as the preferred level. If reliability is considered to Recordings of aggressive behavior by two independent observers in each of 12 intervals. A ✓ indicates that at least one aggressive incident was judged to have occurred in that interval. The observers were in disagreement only in intervals 4 and 5, yielding an interobserver reliability score of 10/12 = .833 = 83.3%. Table 2.1 INTERNAL 1 2 3 Observer A ✓ ✓ Observer B ✓ ✓ 4* ✓ 5* 6 ✓ 7 8 9 10 11 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 12 65 66 Chapter 2 Research Methods Quick Quiz D be inadequate, then our behavioral definition may need to be revised or our observers may need further training in identifying aggressive incidents. (For additional information on assessing reliability, see Kazdin, 2011.) 1. Behavioral definitions should be (subjective/objective) as well as (ambiguous/ unambiguous). 2. The force with which a person can squeeze a device that measures grip strength is a measure of i . 3. How long it takes a musician to play a musical piece from beginning to end is a 4. 5. 6. 7. 8. 9. , whereas the number of hours the musician measure of . The amount of practices each week is a measure of time it takes the musician to commence playing following the conductor’s cue to . begin is a measure of The exact manner in which Jana lifts a dumbbell and presses it overhead is called of the behavior. the t The number of fish a person catches in a one-hour period is the measure of the at which fish are being caught. r Recording whether Rodrigo hiccups within each of a continuous series of recording, five-minute time periods is an example of whereas recording whether hiccupping occurs during randomly selected five-minute periods throughout the day is an example of recording. A device commonly used to measure the ongoing rate of a behavior is a r . On this device, a flat line c indicates (no/slow/fast) responding, while a steep line indicates a rate of rate of responding and a shallow line indicates a responding. Which of the following is an example of a rate measure of writing? (a) Number of words written; (b) number of words written per hour. In an interval recording procedure, our two classroom observers agreed on whether or not Mika was being disruptive during 15 of 20 observed intervals. The level of r in this example is %. For i % is considered the minimally acceptable level while this type of measure, % is preferred. 2.3 Research Designs Deciding how to measure a behavior is only part of the problem. We must also determine which method to use to assess the impact of certain variables on that behavior. Several methods are available, and they can be divided into two general types: descriptive methods and experimental methods. Descriptive Research Descriptive research involves gathering information about a behavior and the circumstances within which it occurs. It does not involve the manipulation of! any variables. Descriptive methods include the survey approach, 2.3 Research Designs in! which! individuals answer a series of questions, and the case study approach, which involves the intensive study of a single individual, such as someone who has a rare type of disorder. The descriptive method most relevant to behavioral research, however, is naturalistic observation, in which one systematically observes and records the occurrence of a behavior in its natural environment. Note the word systematically. We are not talking here about casual observations, which may be strongly biased by the researcher’s preconceptions. Behavioral scientists have as many preconceptions about behavior as the average person does—perhaps more, because it is their job to study behavior—and are therefore susceptible to viewing behavior from a biased perspective. To avoid, or at least minimize, such biases, researchers attempt to define their variables precisely and make their observations in a consistent and uniform manner. Jane Goodall’s systematic study of chimpanzee behavior in the wild is a classic example of naturalistic observation. Through her detailed observations, we now know that chimpanzees eat meat (they sometimes kill and devour monkeys), use primitive tools (they sometimes dip a twig into a termite hill to capture termites for food), and engage in warfare (chimpanzees from one group have been observed stalking, attacking, and killing members of a neighboring group; see Goodall, 1990). Naturalistic observation is a commonly used approach in ethology (or what is now often called behavioral ecology), a branch of zoology that focuses on the study of inherited behavior patterns in animals. Such patterns have presumably evolved to help the animal cope with certain aspects of its natural environment. For this reason, inherited behavior patterns are usually best studied within the natural environment (or a close approximation to it), because the behavior may not occur when the animal is removed from that environment. Displays of dominance and submission, for instance, may not be evident unless an animal is allowed to freely interact with members of its own species. For example, a dog’s behavior of rolling over on its back and displaying its underside can be more clearly seen as a submissive gesture when dogs interact with each other than when they interact with us. One of the authors first realized this when he witnessed the family dog being attacked by a much larger dog. Following a brief skirmish, Trixie rolled over on her back and displayed her underside, the same behavior she often displayed toward us. What had always seemed like a simple request for a tummy scratch also functioned as an inborn gesture of submission. Although naturalistic observation is ideal for studying inherited behavior patterns, it also contributes to our understanding of learning. A famous example of this is the “cultural adoption” of food-washing behavior among a troop of macaque monkeys off the coast of Japan. When one monkey acquired the habit of washing sand off a sweet potato by dipping it in lake water (the researchers had left the potatoes on a sandy beach to attract the monkeys to that area), other monkeys in the troop soon imitated this behavior. Interestingly, the oldest monkeys in the troop never adopted this “newfangled way” of cleaning food (Kawamura, 1963). 67 68 Chapter 2 Research Methods Quick Quiz E The naturalistic observation approach is excellent for gaining rich, detailed information about a behavior and the circumstances in which it typically occurs. The main problem with this approach is that it often leaves us uncertain as to which variables affect the occurrence of the behavior; in other words, it is difficult to determine cause-and-effect (or functional) relationships. For example, if you study childhood aggression by observing children interacting on a playground, you may see many displays of aggressive behavior (e.g., grabbing a toy away from another child, pushing, yelling). However, it will be difficult to determine why these aggressive behaviors are occurring. You might not know, for example, whether an aggressive child has had a long history of aggression, is experiencing considerable frustration that day, or has had frequent exposure to violence in the home. In a sense, the natural environment is a vast sea of variables, and sorting out which variables are responsible for which behavior can be a daunting task. Thus, the naturalistic observation approach is often insufficient for gaining a full understanding of a behavior and the variables that influence it. 1. Shannon and her students spend each spring and summer in the local river valley studying the vocalizations of squirrels. If they only observe and record occurrences of the behavior and the circumstances in which it occurs, they are engaging in o method of research, which is the n research. a type of d 2. Shannon clearly defines the behavior they are studying and carefully assesses the level of interobserver reliability among her students. By doing so, she is , in attempting to minimize the possibility of researcher b which the beliefs and expectations of the researchers can unduly influence their observations. 3. The major limitation of the naturalistic observation approach to research is that it is often (or difficult to determine f -and-effect) relationships. Experimental Research Although naturalistic observations can provide detailed information about behavior, they usually do not allow us to draw firm conclusions about the causes of a behavior. To draw such conclusions, it is usually necessary to conduct an experiment. Thus, in their quest to discover functional relationships between environmental events and behavior, behavioral researchers have a strong preference for the experimental approach to research. In experimental research, one or more independent variables are systematically varied to determine their effect on a dependent variable (i.e., the behavior you suspect will change as a result of changes in the independent variable). Any differences in behavior across the different conditions of the experiment are presumed to be caused by the differences in the independent variable. 2.3 Research Designs Behavioral researchers use two main types of experimental designs: group designs and single-subject designs. As will be seen, each type of design has its advantages and disadvantages, and the decision to employ one method or the other largely has to do with the nature of the issue being investigated. Group Designs The most common type of experimental design is the group design, in which researchers manipulate one or more independent variables across groups of subjects. For example, in a simple control group design, individuals are randomly assigned to either an experimental (or treatment) group or a control group; the experimental group is then exposed to a certain manipulation or treatment, while the control group is not. Imagine, for example, that 20 rats are randomly assigned to either an experimental group or a control group. Rats in the experimental group are individually placed in an experimental chamber for 30 minutes, during which time they receive a free food pellet every minute. The rats in the control group are treated exactly the same except they receive no food during the 30-minute session. They are simply allowed to explore the!chamber. The rats in each group receive one session per day for 10 consecutive days. On day 11, a mechanical lever is placed in each chamber, and the rats must learn to press the lever to obtain food. The question of interest is whether the rats that previously received free food will learn to press the lever more readily or less readily than the rats that did not receive free food. Thus, the independent variable in this experiment is the presence versus absence of free food during the initial phase of the experiment, and the dependent variable is the average amount of time it takes for the rats in each group to learn to press the lever for food. (By the way, research has shown that animals that receive free food subsequently have more difficulty learning how to respond for food [Welker, 1976; Wheatley, Welker, & Miles, 1977]. This suggests that exposure to free reinforcers can sometimes impair an animal’s ability to learn how to respond for reinforcers.) Group designs are usually more complicated than the simple control group design we just described. In a factorial design, one examines the effects of two or more independent variables (or factors) across groups of subjects. For example, we might wonder if the damaging effects of free food on ability to learn are dependent on age (a quasi-independent variable, as described earlier in the chapter). Thus, we might rerun this experiment with groups of old rats, middle-aged rats, and young rats. This approach would yield what is known as a 2 " 3 factorial design, in which there are two independent variables (food and age), the first of which has two levels (free food versus no food) and the second of which has three levels (old age versus middle age versus young age). This experiment would therefore involve a total of six groups (old with free food, old with no food, middle-aged with free food, middle-aged with no food, young with free food, and young with no food; see Table 2.2). If the results indicate that free food affects learning ability only in rats of a certain age, then we say that there is an interaction between the effects of free food and age. Such interaction effects give us a finer understanding of the variables in which we are interested, and research is often designed to search for such effects. 69 70 Chapter 2 Research Methods Table 2.2 Six experimental conditions (groups of participants) in a 2!3 factorial experiment involving two levels of a “food” variable (no food and free food) and three levels of an “age” variable (young, middle-aged, and old). YOUNG (Y) MIDDLE-AGED (M) OLD (O) No food (NF) NFY NFM NFO Free food (FF) FFY FFM FFO A variant on a group design that is used in certain types of animal research is a comparative design. A comparative design is a type of group design in which species is used as a quasi-independent variable. It is often used to test evolutionary hypotheses regarding the differences in selective pressures for a particular trait between species. For example, if you hypothesize that rats have evolved to deal with small, enclosed environments better than dogs have (or that dogs have evolved to deal with large, open environments better than rats have), you could examine how quickly dogs and rats learn to find food in an enclosed maze versus a large open area. This is a 2 " 2 factorial design in which there are two independent variables (species and environment), each of which has two levels (rat versus dog; maze versus open area). Of course, you would not use exactly the same apparatus for a rat that you would for a dog; rather, you would attempt to equate the equipment to the relative size and other important traits of each species. This includes recognizing that different species may exhibit certain abilities in very different ways. For example, in a comparison of dogs versus hedgehogs in their ability to learn to avoid danger, one must consider the fact that dogs and hedgehogs cope with danger in different ways, with dogs tending to run away whereas hedgehogs tend to roll up into a ball and freeze. Group designs are excellent for assessing the average effects of certain variables. Cause-and-effect conclusions are possible due to the strict control over the environment that allows the experimenter to rule out alternative explanations. Because all subjects receive identical experiences except for the independent variable that is being manipulated, we can be fairly confident that differences between groups in performance might be the result of differences in the independent variable. Random assignment of subjects to each condition, where suitable, also ensures that various characteristics of the subjects in each group are likely to be evenly distributed across the different conditions. Thus, the different groups across conditions will be pretty much alike at the onset of the experiment, and any differences found at the end of the experiment will most likely be due to our manipulation of the independent variable. Group designs, however, are not without their drawbacks. To begin with, this type of design usually requires a large number of subjects (often 10 or more per group). In fact, for statistical reasons, the larger the number of subjects in a group, the more trustworthy the results. But what if you wished to conduct research on the effectiveness of a behavioral treatment for one individual? It would be impractical to conduct an experiment with a large 2.3 Research Designs 71 number of subjects just to determine if a certain treatment might be effective for one person. Group designs are therefore not well suited for investigating the effect of a certain treatment on a particular individual. A second difficulty with group designs is that they typically focus on the average performance of all subjects in each group. Little attention is given to the performance of individual subjects, even if some subjects differ markedly from the average. For example, going back to our rat study, suppose that 2 out of the 10 rats previously given free food learned to press the lever almost immediately, while the others took much longer. Even if, on average, the rats in this free-food group learned more slowly than the rats in the no-food group, what about the two quick learners? Should we regard them as mere aberrations (“I guess some rats are just brighter than others”), or is it the case that exposure to free food actually facilitates learning in some individuals? Using similar reasoning, one behavior analyst (himself a victim of brain cancer) has questioned the heavy reliance on large-scale clinical trials to determine the efficacy of new drugs in treating cancer. His argument was that a drug that appears ineffective for the average patient might nevertheless be effective for a subset of patients (Williams, 2010). By ignoring individual data, we might never consider such a possibility. In other words, group data is combined to produce a statistical average, but the scores of some individuals within the group may deviate markedly from this average; therefore, average results may have little relevance to particular individuals. (Question: What implication does this have for the value of the advice given during the radio call-in show in the vignette presented at the start of this chapter?) A third limitation of group designs is that the results are often analyzed and interpreted only at the end of the experiment, rather than during the experiment. In some situations, this may be undesirable. If, for example, we are treating a child for self-injurious behavior, we need to be aware throughout whether our treatment is having a positive effect. If the effect is positive, we can maintain the treatment; if there is no effect, or if the effect is negative, we should halt our treatment and try something different. By contrast, group designs that measure effects only at the end of the study usually do not provide us with this type of flexibility. In conclusion, although group designs are excellent for assessing the average effects of independent variables among large numbers of subjects, there are limitations. Alternative designs that do not suffer from these limitations—but have their own limitations—are single-subject designs, which we discuss in the next section. variable is systematically manipulated variable. to determine its effects on a(n) 2. In a behavioral experiment, the variable is usually a change in some type of environmental experience and the variable is a change in behavior. 3. In the simplest form of a control group design, individuals are r (or tr ) assigned to either an e group. group or a Quick Quiz F 1. In an experiment, a(n) 72 Chapter 2 Research Methods 4. Group designs that are used to assess behavioral differences between species are designs, in which different species constitute called c variables. one of the 5. Briefly stated, three problems with control group designs are: a. b. c. Single-Subject Designs Unlike group designs, single-subject designs require only one or a few subjects to conduct an entire experiment. Also known as single-case or small-n designs, there are several types of such designs, four of which are described here. Simple-Comparison (AB) Design In a simple-comparison design, behavior in a baseline condition is compared to behavior in a treatment condition. Suppose, for example, that Robert wishes to cut down on smoking (as a first step toward quitting) and wonders if he might be able to do so by punishing himself. In a self-punishment procedure, people apply an aversive consequence to themselves each time they engage in an unwanted target behavior. Self-punishment of smoking might consist of Robert giving his buddy a dollar for each cigarette he smokes. (Another way of looking at this is that Robert has implemented a fine or tax on himself to try to reduce the amount he smokes.) The first step in the program would be for Robert to take a baseline measure of the number of cigarettes he typically smokes each day. The baseline of a behavior is the normal frequency of the behavior that occurs before some type of intervention. Robert could, for example, keep an index card tucked inside the flap of his cigarette pack and make a check mark on it for each cigarette he smokes prior to implementing his self-punishment program. The baseline period should last several days (two weeks is often recommended) to provide an assessment of the typical frequency of Robert’s smoking. If it appears that there is a gradual upward or downward trend in the amount smoked during baseline—sometimes the mere act of closely monitoring a behavior can result in some improvement, a process known as reactivity (refer to Study Tip)—Robert should continue the baseline phase until the behavior stabilizes. Following the baseline, he should then institute the self-punishment procedure for several days. If the treatment is effective, the frequency of smoking during the treatment period should be consistently lower than it was during the baseline period (see Figure 2.3). In this type of study, the baseline period is called the A phase, the treatment period is called the B phase, and the design is sometimes referred to as an AB design. (Students sometimes think the baseline phase is the B phase because 2.3 Research Designs Figure 2.3 Simple-comparison (AB) design. Hypothetical results using a simple-comparison design to assess the effectiveness of a treatment (selfpunishment) on number of cigarettes smoked. The dashed vertical line divides the baseline condition from the treatment condition. Results are consistent with, but do not provide strong evidence for, the notion that the treatment was effective. Baseline (Phase A) Number of cigarettes smoked 20 Treatment (Phase B) 15 10 5 0 0 2 4 6 8 10 12 14 Days the word baseline starts with a B. Don’t think that way. The baseline phase is the A phase because that is the phase we start with, and the treatment phase is the B phase because it follows the A phase.) The major problem with the simple-comparison design is that it does not control for the possibility that some other event occurred at the same time that the treatment was implemented, and it was this other event that caused the change in the behavior. For example, perhaps Robert caught a cold at the same time that he began self-punishment, and it is actually the cold that accounts for the reduction in smoking. The simple-comparison design does not allow us to assess this possibility and thus constitutes a poor experimental design. In other words, it does not clearly demonstrate a functional relationship between the independent variable (self-punishment) and the dependent variable (smoking). At best, it provides only suggestive evidence that the treatment was effective. If, however, you are simply interested in obtaining some type of improvement in the behavior, then a simple-comparison design may be sufficient. (Note that a simple-comparison design is completely different from the “comparative design” that was discussed earlier in this chapter.) Reversal Design A much better design is a reversal design, which consists of repeated alternations between a baseline phase and a treatment phase. If the behavior systematically changes each time the treatment is instituted and later!withdrawn, then a functional relationship has been demonstrated between the treatment and the behavior. In Robert’s case, he would begin with the baseline phase, then institute a self-punishment phase, then revert!to!baseline, and then revert to self-punishment. If the results are 73 74 Chapter 2 Research Methods Study Tip: Reactivity is a change in behavior that often occurs when a!behavior is being observed or somehow monitored, even by oneself. The!change that occurs is often in a positive direction, which means that self-monitoring can sometimes, by itself, produce an improvement in behavior. Unfortunately, the reactivity effect is often temporary, with much of the improvement eventually disappearing. Nevertheless, one can find examples of a more enduring effect, resulting in a long-lasting improvement in behavior. For example, I (one of the coauthors of this textbook) once counseled a college freshman who was feeling overwhelmed by all the work she had been assigned in her classes that fall. Among other things, I suggested that she track her study behavior each day (start times, stop times, and total time studied) using a simple recording sheet. We met only a few times after that as she quickly got on track with her courses. Interestingly, though, I came across that same student a few years later when she was studying in the library. Much to my surprise, she had beside her a photocopy of the recording sheet I had given her years earlier when she had come to see me. She told me that she had used it to track each and every study session throughout her undergraduate program, claiming that it played a major role in keeping her motivated. Although this is only anecdotal evidence and she was no doubt doing many other things right—she graduated with excellent marks and eventually went on to graduate studies—it is not unusual to find examples of highly successful people who claim to have found the act of tracking their behavior to be extremely useful. Skinner (1987) is a prime example in this regard, for years having carefully recorded the amount of writing that he did each day (see also Epstein, 1997). Many famous novelists, such as Earnest Hemingway, have done likewise, carefully tracking their daily output to ensure that they maintain a consistent pace (Watson & Tharp, 2014). As noted in the appendix, self-monitoring is a basic aspect of a properly conducted self-management program. But even if you’re not conducting such a program, you might nevertheless find it helpful, at least occasionally, to track your study behavior or any other behavior you wish to alter. If nothing else, it will make you aware of where the behavior is at, which you can then work to improve. But it’s also possible that the mere act of monitoring, by itself, will result in some degree of improvement. (A word of advice: You are much more likely to consistently monitor your behavior if you make it as easy [i.e., as effortless or “frictionless”] as possible. As indicated in the appendix on self-management at the end of this textbook, the amount of effort it takes to carry out a task can have a surprisingly large effect on whether we will consistently perform that task.) something like those depicted in Figure 2.4, with smoking decreasing each time the treatment is implemented and increasing each time the treatment is withdrawn, then we have obtained fairly strong evidence that the treatment is the cause of the improvement. It is unlikely that some other event, such as illness, coincided precisely with each application of the treatment to produce such systematic changes in behavior. A reversal design is sometimes also called a “withdrawal design,” or an ABA or ABAB design, depending on the number of reversals carried out. The reversal design has many strengths. First, unlike the control group design, it allows an entire experiment to be conducted with a single subject. 2.3 Research Designs Figure 2.4 Reversal (ABAB) design. Hypothetical results using a reversal design to assess the effectiveness of a treatment (self-punishment) on number of cigarettes smoked. The systematic change in smoking across the alternating conditions provides strong evidence that the treatment was the cause of the improvement. Baseline Number of cigarettes smoked 20 Treatment Baseline Treatment 15 10 5 0 0 7 14 21 28 Days As such, the reversal design is often ideal for determining the effectiveness of a behavioral intervention for a particular person. Second, some behaviorists argue that statistical tests are not needed to determine if the changes in behavior are meaningful (Sidman, 1960). One can often just “eyeball” the graph to see if the treatment is working, especially when using guidelines that have been devised for judging whether the data points across phases are sufficiently distinct from each other (Cooper et al., 2020). The underlying logic is that if the results are not clear enough to be judged meaningful by visual inspection alone, then the treatment should be altered to produce a stronger effect. This forces the investigator to attain precise control over the variables influencing the target behavior and to strive for powerful treatments that produce large effects (Michael, 2004). You might be wondering whether results from a reversal design can be generalized to other subjects, since we have demonstrated the effect with only one subject. This is an important question, because in science we are concerned with finding effects that have generality. Behavior analysts, however, do not consider this to be a problem in that these types of studies are typically conducted with more than one subject. Since each subject in the study constitutes an entire experiment, each additional subject constitutes a replication of that experiment. And insofar as replication (or reproducibility) is considered a key factor in establishing the validity of a scientific finding, then finding the same pattern of results for all of the subjects exposed to these procedures would indicate that the results of the experiment are probably valid and have good generality. For example, if we tried the self-punishment treatment with four individuals and they all showed consistent decreases in 75 Chapter 2 Research Methods smoking during the treatment phase, then it is quite likely that this treatment will be effective for many other individuals (although the nature of the punishing consequence might have to be tailored to each individual; what is punishing for one person might not be punishing for another). Thus, studies that use reversal designs are often conducted with four or more individuals to assess the extent to which the findings can be replicated across individuals. For radical behaviorists in particular, this type of replication is considered a strong indicator of the generality and validity of one’s findings. In fact, many of the principles discussed in this textbook, which have proven to be highly replicable, were discovered through single-subject experiments involving relatively few animals and without the use of any type of statistical analysis to determine if the results were “statistically significant.” It is also possible to use a reversal design to assess the effectiveness of more than one treatment. For example, imagine that Robert’s initial treatment turns out to be relatively ineffective and results in little if any improvement. Rather than withdrawing the treatment and returning to baseline, a better strategy would be to implement a new treatment and see if it produces a stronger effect. In this case, Robert might decide that the $1 fine for each cigarette smoked is insufficiently punishing and that the fine should be $2. The implementation of this larger punisher constitutes a new phase of treatment, phase C. If after a week that treatment appears to be successful, Robert can revert to the baseline for a week and then return again to the treatment to confirm its effectiveness. This would then be called an ABCAC design (see Figure 2.5). Figure 2.5 Two-treatment reversal design. Hypothetical results in which a reversal design was used to assess the effectiveness of two treatment procedures on the number of cigarettes smoked. When the first treatment (B) produced little improvement, a second treatment (C) was implemented. This treatment was then alternated with a baseline period to confirm its effectiveness. This would therefore be called an ABCAC design. Baseline Treatment 1 20 Number of cigarettes smoked 76 Treatment 2 Treatment 2 Baseline 15 10 5 0 0 7 14 21 Days 28 2.3 Research Designs Figure 2.6 Reversal (ABAB) design. Hypothetical results in which a reversal design was used to assess the effectiveness of a self-punishment treatment on smoking. In this case, the behavior did not revert to its baseline level when the treatment was withdrawn. Thus, although it is possible that the treatment was the cause of the improvement, these results do not provide strong evidence in that regard. Baseline Number of cigarettes smoked 20 Treatment Baseline Treatment 15 10 5 0 0 7 14 21 28 Days Reversal designs have some advantages, but they also have some disadvantages. The first disadvantage is that if the behavior doesn’t revert to its original baseline level when the treatment is withdrawn, we will be left wondering if the treatment was effective. If, for example, the results for Robert’s study looked something like those depicted in Figure 2.6, in which smoking does not return to its pretreatment level during the reversal to baseline, we would be in no better situation than if we had run a simple-comparison design. Although the rate of smoking dropped when Robert first instituted the selfpunishment procedure, it did not climb back up when the procedure was halted; therefore, we cannot be sure that self-punishment was the actual cause of the initial decrease. Although we may be pleased that Robert is now smoking less than before, from a scientific perspective of demonstrating the effectiveness of self-punishment, these results are less than ideal. A second and related disadvantage is that the design is inappropriate for situations in which the treatment is intended to produce a long-lasting or permanent change in behavior such that you would not expect to see a reversal. For example, a student who is being taught math in a particular way will hopefully experience a permanent increase in math ability. We don’t expect their newfound math ability to disappear when the teaching ends. A reversal design is therefore not appropriate for assessing the effect of an intervention like teaching because the improvement is expected to remain long after the intervention has ended. A third disadvantage with a reversal design is that it may, in some circumstances, be ethically inappropriate to remove a treatment once some!improvement has been obtained. If, for example, the implementation of a!treatment results in the elimination of a person’s severe drug addiction, 77 78 Chapter 2 Research Methods Quick Quiz G is!it reasonable for us to temporarily withdraw the treatment in the hope that the addictive behavior will reappear? Although from a scientific perspective removing the treatment would help confirm its effectiveness, such removal would not be ethical. In such cases, we must instead look for another method of demonstrating a functional relationship between the implementation of the treatment and the improvement in behavior. One alternative is to use a multiple-baseline design. 1. In a simple-comparison design, behavior in a b condition is compared to behavior in a t condition. 2. A simple-comparison design (does/does not) allow us to determine if there is a relationship between the independent and f dependent variables. 3. A reversal design, also called a(n) design, involves period and a(n) repeated alternations between a(n) period. 4. What type of result do we need to see during the second baseline phase to determine whether our treatment is the cause of the change in the behavior? 5. A reversal design is not appropriate for an experiment in which the treatment is expected to produce a (temporary/permanent) change in the behavior even after the treatment is withdrawn. 6. A reversal design is also not appropriate when the act of removing a successful problems. treatment would lead to e Multiple-Baseline Design In a multiple-baseline design, a treatment is instituted at successive points in time for two or more persons, settings, or behaviors. As an example of a multiple-baseline-across-persons design, imagine that we have three people who wish to try a self-punishment program for smoking. We begin by taking a baseline measurement of smoking for each person. At the end of the first week, we have one person begin the treatment, while the other two carry on with the baseline. At the end of the second week, we have a second person begin the treatment while the third person carries on with the baseline. Finally, at the end of the third week, the third person also begins the treatment. Thus, across the three individuals, the treatment is implemented at different points in time. If the improvement in behavior coincides with the implementation of the treatment for each individual, then a functional relationship between the treatment and the improvement in behavior has been demonstrated (see Figure 2.7). As an example of a multiple-baseline-across-settings design, imagine that the three graphs in Figure 2.7 represent Robert’s rate of smoking in three different settings: at work, at home, and at the coffee shop. After a week of baseline, Robert begins self-punishing his smoking, but only at work. After the second week, he begins self-punishing smoking at home while continuing to punish it at work. Finally, after the third week, he also starts punishing his smoking behavior at the coffee shop. If his rate of smoking in each setting 2.3 Research Designs 79 Figure 2.7 Multiple-baseline design. Hypothetical results using a multiplebaseline-across-persons design to assess the effectiveness of a treatment (selfpunishment) on number of cigarettes smoked. The three graphs represent the data for three different persons. For each person, the improvement in behavior coincides with the point at which the treatment was implemented. This result shows a functional relationship between the treatment and the improvement in behavior. Person A 25 Baseline Treatment 20 15 10 5 0 Person B Number of cigarettes smoked 25 20 15 10 5 0 Person C 25 20 15 10 5 0 0 7 14 Days 21 28 80 Chapter 2 Research Methods Quick Quiz H drops only when the self-punishment procedure is implemented there, then the procedure is highly likely to be the cause of the improvement. As an example of a multiple-baseline-across-behaviors design, imagine that the three graphs in Figure 2.7 represent three of Robert’s problem behaviors —for example, smoking, swearing, and nail-biting. In this case, we implement the treatment at different times for each behavior. If each behavior shows improvement only when the treatment is implemented, then we have again demonstrated a functional relationship between the treatment and behavior. The multiple-baseline design is a good alternative to the reversal design in that we do not have to worry about withdrawing the treatment to determine that it is effective. This design is therefore appropriate for situations in which the treatment is likely to produce a permanent change in behavior, or in which it may be unethical to withdraw the treatment once some improvement has been achieved. Nevertheless, this design is limited because we need to have more than one person, setting, or behavior to which the treatment can be applied. A second limitation is that the treatment effect might generalize across the different settings or behaviors before the treatment is instituted within those settings or behaviors. For example, as Robert begins to exert more control over his smoking at work, this effect might generalize to his smoking patterns at home and at the coffee shop even before the treatment is applied in those settings. Under such circumstances, it would be difficult to determine whether the treatment was in fact the cause of the improvement. 1. With a multiple-baseline design, the treatment is instituted at different points f o r o n e o r m o re p , in t , or b . s 2. A key advantage of the multiple-baseline design is that we do not have to the treatment to determine if it is effective. w 3. It is therefore a preferable design for situations in which the treatment might result in a (temporary/permanent) change in behavior or where it might be to withdraw the treatment. un Changing-Criterion Design In some circumstances, the treatment is not intended to produce a large, immediate change in behavior but rather a gradual change over time. A useful design for measuring such changes is a changing-criterion design, in which the effect of the treatment is demonstrated by how closely the behavior matches a criterion (or standard) that is being systematically altered. Imagine, for example, that Robert decides to use self-punishment to gradually reduce his smoking behavior. Following a baseline period, he sets a certain criterion for an allowable number of cigarettes that is slightly less than the average number of cigarettes he smoked during the baseline. If!he successfully meets this criterion for three consecutive days, he reduces the allowable limit by two cigarettes. If he meets that criterion for three consecutive days, he reduces the limit by two more cigarettes. He repeats this process until the eventual goal of no smoking has been achieved. The!self-punishment procedure consists of 2.3 Research Designs tearing up a dollar bill for every cigarette that is smoked over the allowable limit (see Axelrod, Hall, Weiss, & Rohrer, 1974, for a case report of such a procedure). Hypothetical results for this program are displayed in Figure 2.8. Robert was usually successful in meeting each criterion, with the number of cigarettes smoked per day either matching or falling below the criterion for that day. The four exceptions occurred on days 29, 32, 33, and 34. Because Robert exceeded the criterion on these days, he would have implemented the self-punishment contingency of tearing up a dollar bill. As noted previously, the changing-criterion design is appropriate for situations in which the behavior is intended to change gradually by some specified amount. Thus, it would be an appropriate design for gradually increasing the amount of time one studies each day or decreasing the amount of time spent playing computer games. It is important, however, that the behavior closely match the changing criteria; otherwise, it will be difficult to determine if the change in behavior is the result of the treatment or of some other factor. The design can, however, be greatly strengthened by including periods in which the criteria suddenly change in the opposite direction (e.g., in the case of Robert, the number of cigarettes allowed would sometimes be raised for a few days). If the behavior continues to track the criteria closely even when they change direction, then we will have obtained strong evidence for the effectiveness of the treatment. In a sense, we have created a changing-criterion design that incorporates aspects of a reversal design. Figure 2.8 Changing-criterion design. Hypothetical data using a changingcriterion design to assess the effectiveness of self-punishment to gradually reduce smoking. The dashed horizontal lines indicate the changing criterion for the allowable number of cigarettes smoked per day. Baseline Number of cigarettes smoked 20 16 Treatment criteria (allowable cigarettes per day) 14 12 10 8 6 4 2 0 16 12 8 4 0 0 7 14 21 Days 28 35 81 82 Chapter 2 Research Methods Advice for the Lovelorn Dear Dr. Dee, I am suspicious that my partner is having an affair with his ex. Whenever she’s in town, he texts me significantly less often. For example, between May and August, when I know for a fact that she was in town, he texted me an average of 5.8 times per day, while between September and December, when she was out of town, he texted an average of 6.4 times per day. I worked it out, and sure enough, this is a statistically significant difference! But when I confronted him with this hard evidence of his unfaithfulness, he denied it and said that I’m being paranoid. Am I Being Paranoid? Dear Am I, Given the evidence that you have presented, I would have to say yes, you may be paranoid. Worse than that, you’re being a poor scientist. For example, you have neglected to consider other factors that might account for the observed difference. Quite apart from his ex being in town, your partner may be texting less often between May and August for other reasons, such as spending more time in outdoor activities. Have you controlled for those variables? Such possibilities need to be considered before you can draw any conclusions about his alleged unfaithfulness. You also need to recognize that statistically significant differences do not provide hard evidence of anything. What they provide is supportive evidence for a certain possibility. Even with a highly significant difference between two sets of scores, there is still a slight possibility that the difference is actually due to chance variation. As well, you need to consider that a difference that is statistically significant may not be meaningfully significant. In fact, the difference you have described seems quite small. I bet that if you chart the number of phone calls week by week, as in a simple-comparison design, you won’t see much of a difference between the May–August period and the September– December period. And if you don’t see much of a difference by eyeballing the data, then maybe there isn’t much of a difference. Behaviorally yours, The reversal design and multiple-baseline design are the most basic singlesubject designs—with the reversal design often regarded as the strongest design—while the changing-criterion design is less often utilized. Other types of single-subject designs have also been devised, each having its advantages and disadvantages (see Kazdin, 2011; Cooper et al., 2020). For!example, in 2.4 Use of Animals in Behavioral Research 83 a multi-element (or alternating treatments) design, one randomly alternates between conditions, often on a day-by-day basis, independent of the level of responding. This might, for example, be a useful design for determining whether eating certain foods before bed is the cause of a person’s bad dreams or nightmares. Most of these designs have been developed for use in applied settings. In experimental research, the control group design is often employed in studies of classical conditioning, while the reversal design (or some variant of it) is often employed in studies of operant conditioning, especially by behavior analysts. programs designed to produce a (sudden/gradual) change in behavior. 2. In a changing-criterion design, it is important that the level of behavior (exceed/ closely match/lag behind) the changes in the criterion for that behavior. 3. The changing-criterion design can be strengthened by including periods in which the criterion suddenly 2.4 . Use of Animals in Behavioral Research Animal research has greatly contributed to our understanding and treatment of serious diseases and illnesses as well as to our understanding of basic physiological processes. Similarly, many of the basic principles of behavior have been discovered through research with animals, especially rats and pigeons. But if the ultimate goal of such research is to discover principles of behavior that are applicable to humans, why use animals at all? In this section, we outline some of the advantages and disadvantages of animal research that may help inform your opinion in this highly controversial debate. Two advantages of using animals in research are the ability to control their genetic makeup and their learning history. Knowledge of an animal’s genetic makeup can help us eliminate, or assess, the effects of inherited differences on learning and behavior. Rats, for example, can be bred so that entire batches are genetically identical or highly similar. We can try to control for genetic differences in humans by studying identical twins, but the number of people we can obtain for such research is necessarily quite limited. Similarly, animals bred for research have had somewhat identical experiences during their upbringing, along with a fairly limited learning history. By comparison, it is impossible to control for the learning histories of humans who volunteer for psychological research. If we are conducting experiments designed to assess basic principles of learning, then the learning histories of our subjects could critically influence the outcome of the experiment. A third advantage to using animals as subjects is that researchers are often able to more strictly control the experimental environment for animals than for humans. This advantage is especially important in behavioral research, in which we Quick Quiz I 1. A changing-criterion design is most appropriate for assessing the effect of 84 Chapter 2 Research Methods are attempting to isolate and manipulate certain aspects of the environment to determine their effect on behavior. For example, if we are interested in the effect of food deprivation on activity level in rats (as discussed in Chapter 12), then it is highly advantageous to strictly control the rat’s feeding schedule— to a degree that would be difficult to attain in humans. Human subjects participating in ongoing research also have an unfortunate tendency to discuss the research task with their friends when they leave the lab each day, even when they are asked not to do so. These conversations can easily lead to a significant change in the person’s behavior during the next experimental session. By contrast, rats tend not to give each other suggestions while lounging about in their home cages following a hard day of lever pressing. Their behavior therefore tends to be more consistent from day to day. In general, because animals are more easily isolated from extraneous influences during the course of the experiment, their behavior is more likely to reflect the true influence of the independent variable. A fourth reason for using animals in behavioral research is that some research cannot ethically be conducted with humans. This is particularly the case with experimental manipulations that are potentially aversive or harmful. For example, rats have been used to investigate the manner in which classical conditioning might account for unusual instances of drug overdose (this finding is discussed in Chapter 5). Investigations using such an animal model of drug addiction have the potential to save lives but would be impossible to conduct with human subjects. (An animal model is a procedure that uses animals to mimic a particular human characteristic or symptom, such as drug addiction or anorexia nervosa.) In reaction to these claimed benefits of animal research, critics have offered several counterarguments. One criticism is that because animals are not humans, the findings from animal research necessarily have limited applicability to humans. The physiological processes, genetic tendencies, and learning histories of animals are simply too different for research with animals to be of much relevance to humans. In this text we hope to convince you of the opposite, but the argument should not be dismissed out of hand. It is reasonable to question whether the results of any experiment are generalizable to species beyond the one studied, and it would be irresponsible to assume that results necessarily generalize across species. Despite the demonstrated benefits of animal research, some research with animals almost certainly does have little applicability to humans. Unfortunately, determining ahead of time which research findings are likely to be applicable to humans is a difficult task. Some of the most applicable findings from animal research, such as basic research on schedules of reinforcement (discussed in Chapter 7), initially would have struck some people as trivial and unimportant. (In fact, some people opposed to behaviorism still regard these findings as trivial and unimportant.) Perhaps the most fundamental criticism of animal research is that it is morally wrong and that animals have rights similar to humans. Animal rights activists oppose “inhumane” research practices, such as confining animals 2.4 Use of Animals in Behavioral Research 85 to cages, subjecting them to electric shock, depriving them of food, and so on. From this perspective, even the reported benefits of animal research for saving lives and improving the human condition are insufficient to justify submitting animals to such morally reprehensible practices. (See the And Furthermore box: Cruel Starvation or a Healthy Diet.) Beginning in the 1800s, researchers have reacted to such criticism by developing guidelines that weigh the benefits of research against the injurious or aversive nature of the procedures. The first guidelines were formulated in 1876, with the introduction of the British Cruelty to Animals Act. It was in the 1960s, however, that animal care committees and review boards became strongly established. Today, researchers in most professional organizations, including the American Psychological And Furthermore Cruel Starvation or a Healthy Diet: The Ethics of Food Restriction In many of the animal studies described in this text, food is used as a reward (reinforcer) for performing certain behaviors. As such, the animals are typically food deprived to ensure that they are well motivated to work for food. Pigeons, for example, are typically placed on a diet until their weight is about 80 to 85% of their free-feeding weight, which is the amount they weigh when food is constantly available. Some people regard such food restriction procedures as inhumane. But is this really the case? First, we have to remember that the 80 to 85% level is calculated relative to the pigeon’s weight when food is freely available. Free food is an unnatural state of affairs for a pigeon, which in its natural environment must constantly forage for food. The result is that the weight of a pigeon on free food is well beyond its optimal weight. Poling, Nickel, and Alling (1990), for example, found that wild pigeons placed on free food for 42 days experienced an average weight increase of 17%, with some pigeons gaining as much as 30%. (This latter figure is equivalent to a 160-pound individual who, with little to do but eat, ends up weighing 208 pounds!) Thus, the weight of a pigeon at 80% of its free-feeding weight may be quite close to what it would be if it were foraging for food in its natural environment. A second point to bear in mind is that a certain amount of food restriction can be healthy. People in countries with higher levels of obesity and diet-related illnesses are aware that eating everything in sight is not a healthy pattern of behavior, and most of us therefore self-restrict to some extent. In fact, calorie restriction has been shown to be a reliable means for slowing the aging process. Many species show significant increases in both health status and life span when raised on high-quality diets that provide 30 to 50% fewer calories than when allowed to free feed (e.g., Weindruch, 1996; Koubova, 2003; Mattson, 2012). And humans too seem to benefit from a certain degree of calorie restriction or short periods of calorie restriction (known as intermittent fasting or time-restricted feeding; for example, Mattson, Longo, & Harvie, 2017). (But see also the discussion on activity anorexia in Chapter 12 for a warning of what can happen when calorie restriction is taken too far.) 86 Chapter 2 Research Methods Quick Quiz J Association, are regulated by ethical standards that provide strict guidelines for the care and use of animals. It is also worth noting that animal researchers are themselves concerned about the welfare of their animals. Skinner, for example, disliked shocking rats and therefore conducted few studies of punishment (Bjork, 1993). And researchers generally acknowledge that the animal rights movement has served a valuable purpose by ensuring the development of strict standards of ethical conduct. They likewise recognize that the extent to which animal research is justified is a difficult question that individuals must answer for themselves. The important thing, however, is to make it an informed decision. (For a discussion of these issues, see Smoo & Resnick, 2009.) 1. Two advantages to using animals for behavioral research is that one can more strictly makeup and l control an animal’s g history. 2. A third advantage to using animals is that the ex environment can more easily be controlled for animals than for humans. 3. A fourth advantage to using animals for research is that it would be to conduct certain types of studies with humans, such u as examining the effects of brain lesions on learning ability. 4. Two arguments against the use of animals in research are: a. b. In Review To create your own summary of this chapter, we recommend three or four paragraphs. One paragraph could focus on the purpose of behavioral research as well as provide clear statements about different types of research, variables, stimuli, and responses. A second paragraph could describe the various ways to measure behavior, and a third paragraph could describe the different types of research designs. A fourth paragraph could outline the pros and cons of using animals in research. Another (or additional) way to summarize parts of this chapter would be to create tables that allow you to quickly compare different methods and procedures. For example, you might create one table that organizes information about the different ways to measure behavior, with the name of each method in the first column (rate, intensity, speed, etc.) and other information in subsequent columns (e.g., definition, type of behavior for which it is most appropriate, limitations). Another table could be constructed for comparing the different types of experimental designs. And if you have difficulty understanding a particular type of single-subject design, try creating your own example and then draw a graph to represent what your results might look like. How does your graph compare to the one in the chapter? In general, deciding how best to organize and summarize Study Questions the information in this chapter will very likely greatly enhance your understanding of and memory for this material. Further Exploration Research methods can be challenging topics for people who are new to the field, but understanding how research works can help you to learn more in other classes as well as interpret information in the broader media. If you can read the journal articles for yourself, then you can come to your own conclusions. Here is a quick list of tips for how to read a journal article: www.psychologytoday.com/us/blog/parallel-process/201808 /how-read-scientific-article. If you want to create your own graphs for a behavioral intervention, check out these guidelines from the American Psychological Association: www.apa.org/gradpsych/2009/03/graphics. For more information on how “covert” responses can be made “overt,” this journal article provides details about using neuroscience methods to measure covert activity: www.ncbi.nlm.nih.gov/pmc/articles /PMC3359857/. Study Questions 1. Distinguish between independent and dependent variables. What is a functional relationship? In experimenting on the effectiveness of a certain study technique on test performance, what would be the independent and dependent variable? What would indicate a functional relationship? How is a quasi-independent variable different from an independent variable? 2. Define stimulus and response. When does the word response tend to be used rather than behavior? Differentiate between the terms stimulus and stimuli. 3. Distinguish between overt and covert behavior and between appetitive and aversive stimuli. 4. What is a motivating operation? Name and describe two types of motivating operations. Provide an example of each that makes use of the procedure of deprivation or satiation. 5. Distinguish between contiguity and contingency. How might contiguity affect the perception of a contingency in determining whether a certain food is causing you to break out in a rash? 6. Define rate of response. How does one distinguish between a high rate of response versus a low rate of response versus a period of no response on a cumulative record? 7. What is the distinction between speed, duration, and latency measures of behavior? Describe how each could be used in assessing the behavior of reading a book. 87 88 Chapter 2 Research Methods 8. Define the intensity and topography of a behavior. Describe how each could be used in assessing the behavior of throwing a ball. 9. Describe interval recording and time-sample recording. Describe how each could be used to assess the extent to which the student who is sitting in front of you is looking at their cell phone during the lecture period. What constitutes the overall measure of behavior in these recording procedures? 10. What does it mean to say that a measure of behavior is reliable? How does one calculate interobserver reliability when using an interval or a time-sample recording procedure? What is the preferred versus minimally acceptable level of reliability? 11. What are descriptive research methods? Describe the use of naturalistic observation to assess the foraging behavior of a squirrel. What is the major benefit and the major limitation of such research methods? 12. What is experimental research? Describe the simplest type of group design that could be used to assess the effectiveness of the 3R approach to studying (which is discussed in Chapter 1). What are three limitations of group designs? 13. What are single-subject designs? Describe a simple-comparison design and draw a graph of some hypothetical results using such a design. (For both this and the following three items, make sure that the graph is properly drawn and labeled.) In what sense is the simple-comparison design a “flawed” design? 14. Describe a reversal design and draw a graph of some hypothetical results using such a design. How is it superior to a simple-comparison design? What are three disadvantages of reversal designs? 15. Describe a multiple-baseline design and draw a graph of some hypothetical results using one version of this design. What is the advantage of this design over a reversal design? What are two limitations of this design? 16. Describe a changing-criterion design and draw a graph of some hypothetical results using such a design. For what types of treatments is this design most appropriate? 17. List four advantages and two disadvantages of using animals as subjects in behavioral research. Concept Review abolishing operation. A procedure that decreases the appetitiveness or aversiveness of a stimulus or event. appetitive stimulus. An event that an organism will seek out. aversive stimulus. An event that an organism will avoid. baseline. The normal frequency of a behavior prior to an intervention. changing-criterion design. A type of single-subject design in which the effect of the treatment is demonstrated by how closely the behavior matches a criterion that is being systematically altered. Concept Review comparative design. A type of group design in which different species constitute one of the independent variables. contingency. A predictive relationship between two events such that the occurrence of one event predicts the probable occurrence of the other. control group design. A type of group design in which, at its simplest, subjects are randomly assigned to either an experimental (or treatment) group or a control group. covert behavior. Behavior that can be subjectively perceived only by the person performing the behavior. Thoughts and feelings are covert behaviors. Also known as private events or private behavior. cumulative recorder. A device that records total number of responses over time and provides a graphic depiction of the rate of response. dependent variable. That aspect of an experiment that is allowed to freely vary to determine if it is affected by changes in the independent variable. descriptive research. Research that focuses on describing behavior and the circumstances within which it occurs. duration. The total amount of time that an individual repeatedly or continuously performs a certain behavior, with no concern for how fast or slow the behavior is occurring. establishing operation. A procedure that increases the appetitiveness or aversiveness of a stimulus. experimental research. A research method in which one or more independent variables are systematically varied to determine their effect on a dependent variable. factorial design. A type of group design in which one examines the effects of two or more independent variables (or factors) across groups of subjects. functional relationship. The relationship between changes in an independent variable and changes in a dependent variable; a cause-and-effect relationship. group design. A type of experimental research in which one manipulates one or more independent variables across groups of subjects. independent variable. That aspect of an experiment that is made to systematically vary across the different conditions in an experiment. intensity. The force or magnitude of a behavior. interobserver reliability. The extent to which two or more independent observers agree on the occurrence of a behavior. (Also known as interobserver agreement). interval recording. The measurement of whether a behavior occurs within a series of continuous intervals. (The number of times the behavior occurs within each interval is irrelevant.) latency. The length of time required for a behavior to begin. motivating operation. A procedure that affects the appetitiveness or aversiveness of a stimulus. (There are two types: establishing operations and abolishing operations.) multiple-baseline design. A type of single-subject design in which a treatment is instituted at successive points in time for two or more persons, settings, or behaviors. 89 90 Chapter 2 Research Methods naturalistic observation. A descriptive research approach that involves the systematic observation and recording of behavior in its natural environment. overt behavior. Behavior that can potentially be observed by an individual other than the one performing the behavior (which is also what is normally meant by the term observable behavior). quasi-independent variable. A variable that is inseparable from an individual and cannot be manipulated, but group-level comparisons can be made using that variable. rate of response. The number of responses that occur in a certain period of time (i.e., the frequency of the response). response. An alternative term for behavior, but one that tends to be used when referring to a particular instance of a behavior or a reaction to a stimulus. reversal design. A type of single-subject design that involves repeated alternations between a baseline period and a treatment period. simple-comparison design. A type of single-subject design in which behavior in a baseline condition is compared to behavior in a subsequent treatment condition. single-subject design. A research design that requires only one or a few subjects in order to conduct an entire experiment. (Also known as singlecase or small-n designs.) spatial contiguity. The extent to which events are situated close to each other in space. speed. The length of time required to perform a behavior from start to finish, which indicates how fast or slow the behavior is occurring. stimulus. Any event that can potentially influence behavior. (The plural for stimulus is stimuli.) temporal contiguity. The extent to which events occur close together in time. time-sample recording. The measurement of whether a behavior occurs within a series of discontinuous intervals. (The number of times that it occurs within each interval is irrelevant.) topography. The physical form of a behavior. variable. A characteristic of a person, place, or thing that can change (vary) over time or from one situation to another. Chapter Test 12. Using a(n) recording procedure that lasted 10 minutes, Erik’s classmate observed that he blew his nose at least once during the first and second minute, as well as the fourth, seventh, ninth, and tenth minutes. His overall level of nose blowing is calculated as %. 27. Jamie decides to implement a program to gradually reduce the amount of time they play computer games. A useful design for determining the effectiveness of this program would be a design. Chapter Test 3. Each time it rains, there is an increase in the number of umbrellas being carried on the street. There appears to be a relationship between rain and the appearance of umbrellas. 23. The length of time it takes Raj to read a chapter from start to end is a(n) measure of behavior, while the amount of time it took before he started reading is a(n) measure of behavior. The total amount of time he spends reading each day is a(n) measure of behavior. 28. The reversal design is also known as a(n) design. 15. In a simple group design, subjects are randomly assigned to a treatment or nontreatment condition. 11. Number of cigarettes smoked per day is a(n) measure of smoking. 30. The least relevant measure for assessing how much a person likes to watch television would be (latency/duration/speed). 19. Animals are often used in behavioral research because this practice allows for greater over learning history, genetic influences, and experimental environment than is possible with humans. Furthermore, animals are often used when the use of humans would be . 10. In measuring the force with which a boxer delivers a blow, we are measuring the of the behavior. 26. I wish to test a new drug that I believe will permanently remove the symptoms of a rare neurological disorder. Three patients with the disorder have volunteered to take the drug. What would be a useful type of design to demonstrate the effectiveness of this drug? 16. The strongest type of single-subject design for determining the effectiveness of a behavioral intervention (given that it can suitably be used) is a(n) design. On the other hand, if I’m only interested in obtaining an improvement in the behavior, then a design may suffice. 4. A flash of light is called a , while a beep and a flash of light are called . The general act of lever pressing would typically be called a (response/behavior), while a specific lever press would tend to be called a . 20. After Trish told Jen that Ryan was the nicest guy in school, Jen started spending more time with him. Trish’s statement about Ryan apparently functioned as a(n) operation that increased Ryan’s value as a(n) stimulus. 25. You have just eaten a large pizza. It is likely that the reward value of eating a pizza has now (increased/decreased) as a function of s . This is an example of a type of operation known as a(n) operation. 1. Any characteristic of a person, place, or thing that can differ over time or between individuals is called a . 91 92 Chapter 2 Research Methods 9. Robbie is afraid of spiders, but Tami finds them interesting. A spider is a(n) stimulus to Robbie, and a(n) stimulus to Tami. 24. When people feel confident, they tend to stand straight. In this case, we are using the of a behavior as a measure of confidence. 6. A knife and spoon are placed side by side in a dinner setting, creating spatial (contiguity/contingency) between the two utensils. 18. Dr. Jansen wonders whether the crows that they are studying can solve certain types of problems better than dogs can. In testing this notion, they would use a type of experimental design known as a(n) design, in which species is treated as a(n) variable. 7. Krista has not seen her romantic partner in four months. It is likely that the appetitive value of seeing her partner has (increased/decreased) as a function of d . This is an example of a type of operation known as a(n) operation. 13. Using a(n) recording procedure, a school psychologist unobtrusively observes a class for a 20-minute period four times each day and notes whether some type of disruption occurs during those times. 17. A useful type of group design to determine whether both selective underlining and the 3R approach would result in more effective studying would be a(n) design. 2. In a classical conditioning experiment, one group of dogs first hears a tone and then receives food, while another group of dogs receives food and then hears a tone. Following this, the researcher measures how much the dogs in each group salivate when they simply hear the tone. In this experiment, the order in which tone and food are presented is the variable, and the amount of salivation to the tone is the variable. 22. On a cumulative recorder, (25/75) lever presses per minute would produce a steeper line in comparison to 50 lever presses per minute. A line would indicate a period of time with no lever presses. 14. The two main approaches to behavioral research are the approach and the approach. 29. Two independent observers agree on whether the target behavior occurred during 14 of the observed intervals and disagree on whether it occurred during 6 of the observed intervals. Interobserver reliability is therefore calculated as %, which is (below/at/ above) the minimum acceptable level of reliability. 5. Blinking is a(n) behavior, while thinking about blinking is a(n) behavior. Answers to Chapter Test 8. Dr. Ross studies schizophrenia by watching videos recorded in the common areas of a group home for people who have been diagnosed with that disorder. The residents are going about their normal daily activities and are unaware of the camera. Dr. Ross is using a descriptive research method known as . 21. To determine whether drinking coffee in the evening keeps me awake at night, I observe my sleep patterns for a two-week period during which I drink coffee each evening, followed by a two-week period during which I do not drink coffee in the evening. I am using a design to conduct this study, which will likely give me (strong/questionable) evidence concerning how coffee affects my sleep patterns. Answers to Chapter Test 1. variable 2. independent; dependent 3. functional (or contingent) 4. stimulus; stimuli; behavior; response 5. overt; covert (or private) 6. contiguity 7. increased; deprivation; motivating; establishing 8. naturalistic observation 9. aversive; appetitive 10. magnitude (or intensity) 11. rate 12. interval; (6/10 = .60) 60% 13. time–sample 14. descriptive; experimental 15. control 16. reversal (or ABAB); simple-comparison (AB) 17. factorial 18. comparative; quasi-independent 19. control; unethical 20. establishing; appetitive 21. simple-comparison (or AB); questionable 22. 75; flat (or horizontal) 23. speed; latency; duration 24. topography 25. decreased; satiation; motivating; abolishing 26. multiple-baseline-across-persons 27. changing-criterion 28. ABAB or ABA (or withdrawal) 29. (14/20 = .70) 70%; below 30. speed 93 Chapter 3 Elicited Behaviors and Classical Conditioning At a friend’s party, Uma witnessed her boyfriend flagrantly flirting with another woman. She was initially angry, but when he later apologized for his actions and was very attentive to her, she experienced unusually strong feelings of attraction toward him. Still, she somehow felt manipulated by the whole affair. After all, her friends had warned her that he had a terrible reputation for playing “mind games.” Chapter Outline 3.1 Elicited Behaviors 3.3 Classical Conditioning Reflexes Fixed Action Patterns Pavlov’s Discovery of Classical Conditioning Basic Procedure and Definitions Appetitive and Aversive Conditioning Excitatory and Inhibitory Conditioning Temporal Factors in Conditioning Pseudoconditioning 3.2 Simple Mechanisms of Learning Habituation and Sensitization Opponent-Process Theory of Emotion Learning Objectives 3-1 Define elicited behavior and provide examples of several common types. 3-2 Distinguish between habituation and sensitization, including the contexts that typically facilitate them. 3-3 Use the opponent-process theory of emotion to explain reactions to extreme events. 3-4 Explain the process of classical conditioning, using examples. 3-5 Distinguish between appetitive and aversive conditioning, including the personal nature of whether a stimulus is appetitive or aversive. 3-6 Distinguish between excitatory and inhibitory conditioning. 3-7 Explain how the temporal order of stimuli results in different strengths of conditioning. 94 3.1 Elicited Behaviors 3.1 Elicited Behaviors The word elicit means “to draw out or bring forth.” Thus, an elicited behavior is a behavior that is drawn out by a preceding stimulus. (It is also known as a respondent behavior or respondent. And note that the word is elicit and not illicit, which refers to something illegal, such as an “illicit drug.”) A sneeze produced by a particle of dust or a startle reaction to the sound of a gunshot are examples of elicited behaviors. They are elicited in the sense that they are automatically drawn out by the stimuli that precede them. In this sense, many elicited behaviors are those that we consider to be involuntary. For example, you do not choose to be startled by a gunshot; your startle reaction is an involuntary response to the gunshot. Similarly, you do not choose to salivate when you bite into a lemon; salivating is an involuntary response to the taste of the lemon. In this chapter, we begin by describing different types of elicited behaviors as well as some simple mechanisms by which they can be modified. This will include a discussion of the opponent-process theory of emotion, an intriguing theory that explains a wide variety of emotional phenomena ranging from drug cravings to the sense of loss you feel following a breakup. The remainder of the chapter will then be devoted to introducing the concept of classical conditioning, the first major type of learning to be discussed in this text. Reflexes Reflexes are the most basic form of elicited behavior. A reflex is a relatively simple, automatic response to a stimulus. (It can also be defined as the relationship between such a response and the stimulus that elicits it.) Some reflexes involve only a single gland or set of muscles, such as when you salivate in response to a drop of lemon juice or blink in response to a puff of air. Other reflexes involve the coordinated action of several body parts. For example, the startle response—a defensive reaction to a sudden, unexpected stimulus —involves the automatic tightening of skeletal muscles as well as various hormonal and visceral (internal organ) changes. Similarly, the orienting response—in which we automatically position ourselves to facilitate attending to a stimulus—can involve a relatively major body movement, such as when we automatically turn in response to an unfamiliar noise behind us. Many reflexes are closely tied to survival. For example, food consumption involves a chain of reflexes that include salivation, peristalsis (wave-like actions that push food down the esophagus and through the digestive system), and secretion of digestive juices in the stomach. Conversely, the vomiting reflex serves a protective function by expelling potentially poisonous substances from the digestive system. Other protective reflexes include the flexion response, in which we automatically jerk our hand or foot away from a hot or sharp object that we have inadvertently contacted, and the aforementioned startle reaction, designed to ready us for fight or flight if an unexpected stimulus should prove dangerous. Newborns come “prepackaged” with a host of reflexes that facilitate their survival. For example, if you touch a baby’s cheek with your finger, the baby 95 96 Chapter 3 Elicited Behaviors and Classical Conditioning will automatically turn their head in that direction. This reflex is designed to facilitate taking a nipple into the mouth. Once the nipple is in the mouth, the baby’s sucking reflex is activated (which in turn elicits a “milk letdown” reflex in the breast). Some of these reflexes disappear within a few years (e.g., the sucking reflex), but others, such as salivating and vomiting, remain with us throughout life. Many of the simpler reflexes are activated through a reflex arc: a neural structure underlying some reflexes that consists of a sensory neuron, an interneuron, and a motor neuron, in that order. For example, when you quickly jerk your hand away from an open flame, your flexion response is being activated through a reflex arc. Upon touching the flame, receptors in the hand stimulate sensory neurons that carry a danger message (in the form of a burst of nerve impulses) toward the spinal cord. Within the spinal cord, interneurons receive this message and immediately pass it on to the motor neurons. These motor neurons then activate the muscles in the arm that pull the hand away from the flame. Simultaneously, pain messages are also sent up the spinal cord to the brain; however, by the time they are received and you consciously feel the pain, your hand is already being withdrawn from the flame. Thus, we do not withdraw our hand from the flame because of the pain; we actually begin withdrawing our hand before feeling any pain. Because the flexion response utilizes a simple reflex arc through the spinal cord, we are able to withdraw our hand from the flame much faster than if the message had to be routed all the way through the brain and then back down to the arm muscles (refer to Figure 3.1). Figure 3.1 Schematic diagram of a reflex arc underlying a flexion response. Upon touching a hot or sharp object, sensory neurons carry the message via nerve impulses toward the spinal cord. Interneurons within the spinal cord then receive the message and pass it directly to motor neurons. The motor neurons in turn activate muscles in the arm or leg that carry out the reflex action. At the same time that this reflex arc is being activated, a pain message is sent to the brain. Thus, the reflex arc enables a person to withdraw his or her arm or leg from the damaging stimulus prior to actually feeling any pain. Reflex Arc Sensory Receptors in hand that touches a hot flame Muscles that pull hand away from flame Message to brain (not part of the reflex arc) Sensory Neuron Interneuron in spinal cord Motor Neuron 3.1 Elicited Behaviors stimuli that precede their occurrence. response is an automatic defensive response to response is an a sudden, unexpected stimulus; the o elicited set of movements designed to facilitate attending to a stimulus. 4. Many simple reflexes are activated through a r a that consists of a(n) neuron, a(n) neuron, neuron (in that order). and a(n) 5. Quickly jerking your hand or foot away from contact with an open flame or sharp object response. In such cases, is a reflexive action known as a fl the!perception of pain generally (precedes/follows) the occurrence of the response. 3. A s Fixed Action Patterns Some types of elicited behaviors are more complex than simple reflexes. A fixed action pattern is a fixed sequence of responses elicited by a specific stimulus. Examples include spiders building a web, ducks flying in a V-shaped formation, and some species of squirrels burying nuts. Dogs and cats display numerous fixed action patterns. Cats compulsively scratch the ground to cover up urine and feces (effective in a litter box but completely ineffective on your carpet) and rub up against the legs of visitors to “mark” them as belonging to their territory. Dogs indicate their desire to play by wagging their tails, stretching out their front legs, and lowering their heads to the ground (refer to Figure!3.2). In fact, by adopting this posture (and looking completely foolish in front of any visitors), you can effectively ask your dog if it wishes to play (which of course it will not, given that it has made you look like an idiot). Russell A. Powell Figure 3.2 Example of a fixed action pattern. A dog will indicate its desire for play by stretching out its front legs and lowering its head to the ground. Quick Quiz A 1. A simple, involuntary response to a stimulus is called a . 2. Reflexes are e in the sense that they are drawn out by 97 98 Chapter 3 Elicited Behaviors and Classical Conditioning Quick Quiz B For many fixed action patterns, we are able to identify a specific stimulus that sets it in motion. The specific stimulus that elicits a fixed action pattern is called a sign stimulus or releaser. For example, a male Betta splendens, better known as a Siamese fighting fish, immediately takes an aggressive posture at the sight of another male (the releaser), spreading out its fins and flaring its gills. Similarly, during mating season, a male stickleback fish displays a fixed sequence of aggressive actions when another male enters its territory (Tinbergen, 1951). Interestingly, the sign stimulus for the stickleback’s aggressive actions is not the presence of the other male but the sight of its red underbelly. If the red belly is covered up or painted a different color, the intruder will not be attacked. Conversely, a pie-shaped or cigar-shaped piece of wood with a red patch on the bottom will be attacked. Fixed action patterns tend to be unique to certain species and are therefore sometimes called species-specific behaviors. They can also be called instincts, but some researchers dislike this term because it implies that the behavior is more rigid and inflexible than is actually the case. For example, if two rats are subjected to a painful stimulus, such as an electric shock, they will automatically attack each other (Ulrich & Azrin, 1962). In fact, many species will become aggressive in reaction to pain. In rats, this often takes the form of a fixed action pattern in which the two combatants rear up on their hind legs and, like boxers, strike out at each other with their front paws. Interestingly, this aggression is more likely to occur in rats that have been trained to be aggressive than in those that have not (Baeninger & Ulm, 1969). Thus, the rats’ fixed action pattern of aggression is actually somewhat variable and can be significantly modified by experience. Fixed action patterns are adaptive responses that have evolved to help animals cope with consistent aspects of their environment. The difficulty with such inherited patterns is that a sudden, large-scale change in the environment may render the pattern useless—or even harmful. For example, deer have an inborn tendency to run a zigzag pattern when pursued by a predator. This action, which confuses the predator, greatly increases the deer’s chances of survival in the wild; unfortunately, it also greatly reduces the deer’s chances of survival when it is being pursued by a modern automobile. By comparison, an animal that can modify its behavior patterns through learning can better adapt to a changing environment, which is why the ability to learn was such an important evolutionary advancement. 1. A(n) is a fixed sequence of responses that occurs in reaction to a specific stimulus. The specific stimulus that elicits this behavior is called a s . stimulus or re 2. Different species of spiders spin different kinds of webs. Web spinning of this sort -sp can thus be considered a sp , but some behavior. Such behaviors used to be called i researchers dislike this term because it implies that the behavior is more (flexible/ inflexible) than is actually the case. 3.2 Simple Mechanisms of Learning 3.2 Simple Mechanisms of Learning Habituation and Sensitization The repeated presentation of an eliciting stimulus can alter the strength of the elicited behavior. Habituation is a decrease in the strength of the elicited response following repeated presentations of the stimulus that naturally (without prior learning) elicits that response. For example, a sudden, unexpected tap on the shoulder will naturally elicit a startle response, whereas any additional taps might have no such effect. And we quickly stop attending to low-intensity background noises such as the ticking of a clock or the distant noise of traffic. By contrast, sensitization is an increase in the strength of an elicited response following repeated presentations of the stimulus that naturally elicits that response. For example, soldiers under attack generally do not habituate to the sound of artillery shells exploding nearby. Instead, their startle reaction grows stronger. Needless to say, this greatly contributes to the stress they experience and the inevitable symptoms that most soldiers experience after too much exposure to battle conditions. The effects of habituation and sensitization usually disappear when the stimulus is not presented for a period of time, meaning that the strength of the behavior returns (or recovers) to its original level when the stimulus is next presented. For example, you might habituate to the sound of a neighbor’s stereo one evening, only to be once more bothered by it when they first turn it on the next morning. In the few hours since you last heard the music, your habituation to it disappeared and you again responded to the noise like you normally would. But note that habituation can occur over varying time spans. The examples given so far represent short-term habituation, in which the response quickly decreases as a result of repeated stimulation and the ability to respond then quickly recovers in the absence of stimulation. Conversely, in long-term habituation, the response slowly decreases as a result of repeated stimulation, and the ability to respond then slowly recovers in the absence of stimulation. For example, if you move into an apartment from which you hear the sound of a train each morning, your reaction to the noise (i.e., how much you notice it) will probably be most intense on the first day and slowly decrease thereafter. Moreover, once you become fully habituated to the noise, you would have to be away from your apartment for several weeks or even months before your reaction to the noise would return to its original level. Additionally, long-term habituation tends to occur when presentations of the stimulus are widely spaced (e.g., a train going by your apartment each morning), whereas short-term habituation tends to occur when presentations of the stimulus are narrowly spaced or continuous (e.g., music from the apartment next door). And finally, repeated sessions of shortterm habituation, spread out over time, can eventually lead to long-term habituation. The outside traffic noise that you habituate to each time you come home in the evening eventually becomes unnoticeable even when you first walk in the door. 99 100 Chapter 3 Elicited Behaviors and Classical Conditioning Quick Quiz C Note that sensitization often generalizes to other stimuli. A noisesensitized soldier is likely to jump not only in response to artillery explosions but also to any sudden stimulus. By contrast, habituation tends to be more stimulus specific, such that even small changes in the stimulus may result in the reappearance of the response. Thus, many people suddenly become aware of the sound of their car when the motor sounds a bit different or when the car has a slightly different feel to it as they are driving. Only a minor change is needed to alert the driver that something is potentially wrong (and hopefully, inexpensive to fix). One version of this process is known as the Coolidge effect, based on an old joke about former U.S. president Calvin Coolidge. The story has it that he and his wife were once being separately escorted around a chicken farm. When Mrs. Coolidge was informed that the resident rooster was capable of mating several times a day, she remarked, “You should tell that to the president.” Informed about this, the president asked whether the repeated matings occurred with the same chicken or different chickens. When told that it was different chickens, he said, “You should tell that to my wife.” The Coolidge effect therefore is the enhanced sexual arousal displayed by the males of some species when presented with different sexual partners as opposed to the same sexual partner to whom it has habituated. In what is known as dishabituation, habituated responses to a stimulus can reappear following the presentation of another, seemingly irrelevant novel stimulus. For example, Sherri might quickly habituate to the sound of gunshots at a shooting range. If, however, a stranger approaches and stands nearby, she might again be startled when the next shot is fired. Likewise, couples can sometimes rekindle their romance by traveling to a new and different environment—or even by treating themselves to a night in a hotel room rather than staying at home. 1. An increase in the strength of a behavior following repeated presentations of the eliciting stimulus is called . 2. A decrease in the strength of a behavior following repeated presentations of the eliciting stimulus is called . 3. Learning to ignore the sound of dripping water is an example of ; becoming increasingly aware of the sound of a jackhammer on the street below your . apartment is an example of 4. The fact that it has been several months since you noticed the sound of the morning and evening whistles at a nearby factory is an example of habituation. Such habituation tends to build up (quickly/ t slowly), following which, when the stimulus is absent, the ability to respond recovers (quickly/slowly). This means that you would be most likely to again notice the whistle after returning from a (long/short) vacation. 5. In general, sensitization is (less/more) stimulus specific than habituation. This means that sensitization is (less/more) likely than habituation to generalize to other stimuli. 6. In what is known as dis , the presentation of a (familiar/ novel) stimulus during a period of habituation can sometimes result in the (reappearance/disappearance) of the habituated response. 3.2 Simple Mechanisms of Learning Why does repeated exposure to certain stimuli sometimes result in habituation and sometimes in sensitization? One factor is the intensity of the eliciting stimulus. A low-intensity stimulus, such as the ticking of a clock, typically results in habituation, while a high-intensity stimulus, such as exploding artillery shells, typically results in sensitization. A stimulus of moderate intensity often results in an initial period of sensitization, followed by habituation. For example, the first few shots you hear at a shooting range might produce an increasingly strong startle reaction. But you then begin to habituate to the shots and hardly notice them after a while. Another factor that influences habituation versus sensitization, which can often override the intensity factor, is the evolutionary (adaptive) significance of the stimulus. Habituation and sensitization are processes that we find across species, even in simple organisms like worms and snails (e.g., Wicks & Rankin, 1997). From an evolutionary perspective, this suggests that these processes have tremendous survival advantages. They enable us to categorize stimuli into those that are currently irrelevant, which we tend to habituate to, and those that are currently relevant, which we tend not to habituate to and may even become sensitized to (Eisenstein, Eisenstein, & Smith, 2001). Thus, we find it easier to habituate to the harmless sound of a locomotive shuttling railcars back and forth in a nearby rail yard than the buzzing sound of a wasp in our room that could sting us. Likewise, most people quickly habituate to the smell of onions and spices, even if strong, but become increasingly bothered by the smell of something rotten, which could poison us. This perspective also explains why stimulus intensity can make a difference: low-intensity stimuli are often insignificant, while highintensity stimuli are often highly significant and sometimes dangerous. You may have noticed that we’ve used “gunshot” examples for both sensitization and habituation. Wouldn’t the sound of a gunshot be the same for everyone? The answer is “no.” A person at a shooting range may find the noise startling at first, but has little or no reasonable expectation of danger. Someone in a war zone, in contrast, has a very different expectation about the likelihood of real danger. Our assessment of the risks inherent in the environment can affect our habituation or sensitization in ways that make sense: we’re more likely to sensitize when we perceive danger and more likely to habituate when we perceive that we’re safe. This is an example of how prior learning (or cognitive processes) can interact with more automatic processes to affect new learning (e.g., Sirot, 2010; Mutic, Brünner, Rodriguez-Raecke, Wiesmann, & Freiherr, 2017). Of course, we don’t always get it right and sometimes become sensitized to things that are not dangerous. Wouldn’t you love to be able to habituate quickly to the sound of the barking dog next door? Unfortunately (or fortunately), organisms behave in ways that increase their likelihood of survival and reproduction, which often means erring on the side of caution. Further, there are individual differences in temperament or personality among people and animals that affect the likelihood of habituation or sensitization (LaRowe, Patrick, Curtin, & Kline, 2006; Blanch, Balada, & Aluja, 2014; Allan, Bailey, & Hill, 2020). So, unfortunately, sleepless nights due to barking dogs are a reality for many of us. 101 102 Chapter 3 Elicited Behaviors and Classical Conditioning Quick Quiz D Finally, it is worth noting that habituation and sensitization, although relatively simple processes, have proven to be highly useful in illuminating some of the basic neurological processes involved in learning. For example, Kandel and colleagues (e.g., Hawkins, Bailey, & Kandel, 2010) have carried out extensive investigations into the neural mechanisms that underlie habituation in Aplysia, a large marine snail. When lightly touched, Aplysia withdraws its gill beneath a protective mantle (a “gill-withdrawal reflex”), which habituates with repeated stimulation. Insofar as Aplysia has an extremely simple nervous system—consisting of only a few thousand neurons compared to the many millions of neurons found in even a mouse—it has proven to be an ideal organism for mapping the sequence of neural and biochemical events that lead to habituation. This work in turn has led to key insights into the biological processes that govern learning and memory in general, for which Kandel was awarded a Nobel Prize in 2000. Considerable work has also gone into the neurological processes involved in classical and operant conditioning, which are likewise fundamental forms of learning (e.g., Guerra & Silva, 2010). Thus, although the present text focuses on behavior rather than the biological processes that underlie it, the basic principles and concepts presented herein constitute essential knowledge for anyone interested in the neuroscience of learning and memory. 1. In general, repeated presentations of a low-intensity stimulus result in , and repeated presentations of a high-intensity stimulus . result in 2. A stimulus of intermediate (or moderate) intensity will initially result in a period , which is then followed by . of 3. From an evolutionary standpoint, if a stimulus is irrelevant or “safe,” we tend to to it, whereas if a stimulus is potentially dangerous become we become to it. Opponent-Process Theory of Emotion Habituation and sensitization represent two opposing tendencies: weaker reactivity to a stimulus versus stronger reactivity. Solomon (1980; also Solomon & Corbit, 1974) proposed an intriguing theory of emotion that involves a similar dual mechanism. Known as the opponent-process theory, it is particularly good at explaining the aftereffects of strong emotional responses. Consider, for example, the following anecdote: My neighbor’s son was struck by lightning as he was returning from a golf course. He was thrown to the ground. His shorts were torn to shreds and he was burned across his thighs. When his companion sat him up, he screamed “I’m dead, I’m dead.” His legs were numb and blue and he could not move. By the time he reached the nearest hospital he was euphoric [i.e., extremely happy]. (Taussig, as quoted in Solomon, 1980, p. 691) 3.2 Simple Mechanisms of Learning In one sense, the boy’s euphoria is logical in that he was lucky to be alive. But in another sense, it is illogical because he was injured and decidedly worse off than before the incident. Should he not have remained at least somewhat distressed about the incident? Consider, too, the following scenario. You purchase a lottery ticket during a visit home. The next weekend, your mom calls to tell you the winning numbers—lo and behold, you have won $50,000! Wow! You are absolutely elated. Unfortunately, an hour later, your mom calls you back to tell you that she made a mistake on the numbers, and you have won only $50. You are now extremely disappointed even though you are still $50 better off than when you climbed out of bed that morning. Within a day, however, your disappointment wears off, and you carry on with your impoverished lifestyle as usual. Now consider an experiment in which a dog is exposed to electric shock (e.g., Katcher et al., 1969). During the shock, the dog’s heart rate quickly rises to a peak, decreases slightly, and then stabilizes at a relatively high level. Guess what happens when the shock is turned off. Does the dog’s heart rate return to normal? No, it does not. When the shock is removed, the dog’s heart rate plunges to below normal and then after a few minutes moves back up to normal (refer to Figure 3.3). In fact, the pattern of changes in heart rate during and after the shock—an index of the dog’s emotional response to shock—is very similar to the emotional pattern displayed in the preceding lottery scenario. In both cases, an event elicits a strong emotional response, but when the event is withdrawn, an opposite response is elicited and then gradually disappears. Figure 3.3 Heart rate changes accompanying the application and withdrawal of shock. Our emotional responses often follow a similar pattern, with the onset of the emotional event followed by one type of emotion and the offset of the event followed by an opposite emotion. Shock on Increased heart rate Normal heart rate Decreased heart rate Time Shock off 103 104 Chapter 3 Elicited Behaviors and Classical Conditioning An explanation for such emotional changes is provided by the opponentprocess theory of emotion. The opponent-process theory proposes that an emotional event elicits two competing processes: (1) an a-process (or primary process) that is directly elicited by the event, and (2) a b-process (or opponent process) that is elicited by the a-process and serves to counteract the a-process. For example, the presentation of shock directly elicits a tendency for the dog’s heart rate to increase, which is the a-process. This increase in heart rate in turn elicits a compensatory reaction that tries to decrease heart rate, which is the b-process. The purpose of this compensatory b-process is to counter the!sudden increase in heart rate, thereby maintaining a state of internal balance (known as homeostasis). In other words, the b-process tries to prevent the increase in heart rate from becoming too extreme, which could be damaging or even fatal. The actual heart rate during the shock is therefore the net result of the tendency for heart rate to increase in the presence of shock (the a-process) minus the compensatory tendency for heart rate to decrease (the b-process; refer to Figure 3.4). Think back to the example of the young man struck by lightning: if you knew that one of the body’s reactions to extreme pain is to release pain-killing endorphins (which can also cause euphoria), it makes sense that the b-process (of endorphin release) would compensate for the a-process (pain and fear). Figure 3.4 Opponent-process mechanisms that underlie changes in heart rate due to the onset and offset of shock. During the shock, as the b-process acquires strength, it pulls heart rate down from its initial peak and stabilizes it at a moderately high level. Following shock, when the a-process is no longer active, the b-process pulls heart rate to below normal, then gradually disappears, allowing the heart rate to return to normal. Shock on a-process Increased heart rate (a-process) Net result Normal heart rate b-process Decreased heart rate (b-process) Time Shock off 3.2 Simple Mechanisms of Learning 105 Thus, the young man’s euphoric reaction resulted from the elicitation of a very strong b-process, to the point that it overrode the a-process once the latter started to weaken a bit. Similarly, in the lottery example, the feeling of elation you experience when you think you have won the lottery is the emotional response directly elicited by the thought of winning (the a-process) minus the compensatory reaction to this elation (the b-process), which is trying to keep your elation from becoming too extreme. But when you later discover that you did not win the lottery, all that is left is the compensatory reaction to elation, which is the opposite of elation (i.e., sadness). response is often followed by a(n) (similar/opposite) emotional response. is directly elicited by the , emotional event; this in turn elicits the the purpose of which is to maintain a relatively balanced internal state known as . h 3. The a-process is also known as the pr process, and the process. b-process is also known as the o 2. The The a- and b-processes have some important characteristics: 1. The a-process correlates closely with the presence of the emotional event. This is most clearly evident when the emotional event can be quickly presented and quickly removed. As shown in Figure 3.4, the tendency for the heart rate to increase in response to shock is directly tied to the presence of the shock. When the shock is turned on, heart rate immediately increases; when the shock is turned off, heart rate immediately decreases. Similarly, you immediately become elated when you think you have won the lottery, and your elation immediately disappears when you discover that you have not. 2. The b-process is slow to increase and slow to decrease. The slow buildup in the b-process accounts for why our emotional response to an event is often strongest at the outset. If you consider Figure 3.4 again, you can note that when the shock is first turned on, the dog’s heart rate quickly peaks and then declines slightly before stabilizing. The immediate peak happens during the early moments of shock, when the b-process is not yet strong enough to counteract the a-process, thereby allowing the a-process free rein to increase heart rate. After a few moments, though, the b-process becomes strong enough to moderate the a-process, causing a slight decrease in heart rate before stabilizing. When the shock is removed, the a-process immediately disappears, but the b-process only slowly declines. For this reason, when the shock is turned off, the dog’s heart rate plunges to well below normal, because all that remains is the b-process that has been trying to pull heart rate down. (It is as though the other team suddenly let go of the rope in a tug-of-war, sending your team Quick Quiz E 1. The opponent-process theory of emotion accounts for why a strong emotional 106 Chapter 3 Elicited Behaviors and Classical Conditioning flying backward in the direction in which you were pulling.) Similarly, when you discover that you have not won the lottery, you immediately feel depressed, because the counterreaction to the elation you have been feeling is all that remains. As this b-process gradually weakens, however, your emotional response slowly returns to normal, just as the dog’s heart rate slowly returns to normal. 3. With repeated presentations of the emotional event, the b-process increases in both strength and duration. This is the most interesting part of the theory. For example, what happens to the dog’s heart rate if the dog is repeatedly shocked? As it turns out, the increase in heart rate during each shock becomes less and less extreme. Additionally, each time the shock is turned off, the dog’s heart rate plunges further and further and takes increasingly longer to return to normal (refer to Figure 3.5). The dog’s overt emotional response matches these changes in heart rate. Whereas in the early sessions the dog shows considerable distress in response to the shock, in later sessions it appears more annoyed than distressed. More surprising, though, is the change in the dog’s emotional response following the shock. Whereas in the early sessions the dog Figure 3.5 Effects of repeated stimulus presentations on primary and opponent processes. With repeated stimulation, the b-process (opponent process) becomes stronger and takes longer to disappear. The result is that heart rate rises only slightly above normal during the shock, then drops considerably below normal following the shock and takes a relatively long time to return to normal. Shock on a-process Increased heart rate (a-process) Net result Normal heart rate Decreased heart rate (b-process) b-process Time Shock off 3.2 Simple Mechanisms of Learning appears somewhat relieved when the shock is turned off, in the later sessions it shows signs of extreme pleasure, jumping about and greeting the experimenter with enthusiasm. 4. The a-process and the b-process tend to be hedonically opposite from each other. The word hedonic refers to the extent to which something is experienced as pleasant versus unpleasant. In general, if the a-process is experienced as unpleasant (as shown by the dog’s distress while being shocked), the b-process will be experienced as pleasant (as shown by the dog’s euphoria when the shock was withdrawn). Conversely, if the a-process is experienced as pleasant (as in the elation one feels from having won the lottery), the b-process will be experienced as unpleasant (as in the disappointment one feels in discovering that one did not win the lottery after all). Similar emotional patterns have been found in humans. For example, Epstein (1967) found that military parachutists became less and less terrified with repeated jumps and more and more elated following each jump. This sense of elation can last several hours among veteran jumpers and probably accounts, at least partially, for the strong attraction some people feel toward parachuting and other high-risk activities.1 Other research has suggested that some types of self-injurious behavior (e.g., cutting, hair pulling) may be rewarded by the pleasant feelings that result from the compensatory elicitation of endorphins, which makes treatment of such behaviors quite challenging (Franz, Kleiman, & Nock, 2019). The opponent-process theory of emotion also has implications for a phenomenon known as revictimization (van der Kolk, 1989). Some people repeatedly become involved in abusive relationships or have great difficulty leaving such relationships. A contributing factor may be that the person has become hooked on the powerful feelings of pleasure that occur during the “honeymoon period” of forgiveness that often follows an abusive episode. This intense pleasure is the compensatory afterreaction (the b-process), which has become greatly strengthened during repeated episodes of abuse. As suggested in the opening vignette to this chapter, a weaker version of this honeymoon effect might even occur in relationships in which one is exposed to intermittent periods of emotional distress rather than physical abuse. 1 Note, however, that emotional changes during skydiving can be a bit more complex than this. Veteran skydivers experience a peak of anxiety just before leaving the plane (a highrisk moment), followed by a strong sense of elation during the free fall, another peak of anxiety when the chute is being deployed (another high-risk moment), and then a sense of elation after they land. The strong sense of elation that occurs during free fall might be a contributing factor to accidents because veteran jumpers may be tempted to delay deploying the chute until the last possible moment (Delk, 1980). 107 108 Chapter 3 Elicited Behaviors and Classical Conditioning Quick Quiz F Not all research has supported opponent-process theory (e.g., Newton, Kalechstein, Tervo, & Ling, 2003). Furthermore, as will be discussed in Chapter 5, it appears that classical conditioning often plays a role in the elicitation of opponent processes, especially those associated with drug use. Nevertheless, opponent-process theory has stimulated a considerable amount of research and has proven extremely useful for enhancing our understanding of emotional behavior. 1. With repeated presentations of the emotional event, the b-process (increases/ and d . decreases) in both s is directly tied to the presence of the is (slow/quick) emotional event, whereas the to decrease. to increase and 3. Feeling elated while talking on the phone to someone with whom you are in love ; feeling lovesick after is an example of the . you hang up is an example of the 4. The previous example is also an illustration of how the a- and b-processes are (opposite/similar) to each other; for example, if the h . a-process is pleasurable, then the b-process is 2. The 3.3 Classical Conditioning We have so far discussed those situations in which a certain stimulus (e.g., lemon juice) elicits a particular response (e.g., salivation). We have also noted that repeated presentations of a stimulus can sometimes change the nature of the response, either strengthening it (sensitization), weakening it (habituation), or by eliciting a compensatory reaction (an opponent process). But these are relatively simple means of adaptation that do not involve associating different stimuli with each other or a different stimulus with a different response. They are therefore regarded as examples of nonassociative learning. The world, however, is a complex place filled with a vast array of stimuli. We often need to anticipate whether an event is about to occur and to recognize whether certain events are meaningfully related to other events. For example, when we are first stung by a wasp, it is adaptive for us to associate the pain with the sight and sound of the wasp. It is also adaptive for us to be wary of insects that resemble wasps, because many of them (e.g., honeybees and hornets) also sting. Thus, the ability to relate one event to another, which is known as associative learning, allows us to better anticipate the future, thereby greatly facilitating our chances of survival. The first type of associative learning that we will examine is classical conditioning. 3.3 Classical Conditioning 109 Advice for the Lovelorn Dear Dr. Dee, Several months ago I broke up with my boyfriend when he took a job in another city. We were together five years, and it had turned into a pretty monotonous relationship by the time it ended. But it is definitely taking me a long time to get over it. A friend tells me that I must have some kind of “unresolved dependency issue” to be this depressed. She recently broke up with her boyfriend—she went out with him for only a month but claims that she was madly in love—and got over it in a week. Is there something wrong with me, or is my friend just superficial? Still Depressed Dear Still, There may be several reasons why some people take longer than others to recover from a breakup. The opponent-process theory, however, might be particularly applicable in your case. Remember that our primary emotional response to an event (the a-process) typically weakens with repeated exposure, while the emotional after-reaction (the b-process) typically strengthens. Solomon (1980) suggested that these processes are also applicable to love relationships. Couples who have been together a long time often perceive weaker (or less volatile) feelings of affection for each other than do couples who have been together a short time. When relationships end, however, couples who have been together a long time experience a much deeper and longer-lasting sense of loss than do couples who have been together a short time. According to opponent-process theory, this sense of loss is the compensatory reaction to the relationship, which will be much stronger in those couples who have been together longer. For this reason, it will naturally take more time for you to get over your long-term “monotonous” relationship than for your friend to get over her brief “madly-in-love” relationship. Behaviorally yours, In classical conditioning, a stimulus comes to elicit a response because it has been paired (or associated) with another stimulus. Classical conditioning is also known as Pavlovian conditioning, after Pavlov, who discovered many of the basic principles of classical conditioning. However, behavior analysts who follow Skinner’s approach to behaviorism typically refer to this type of conditioning as respondent conditioning, and the elicited behaviors as respondent behaviors or simply respondents. Chapter 3 Elicited Behaviors and Classical Conditioning Pavlov’s Discovery of Classical Conditioning Ivan P. Pavlov (1849–1936), a Russian physiologist, is generally credited with the first systematic investigations into classical conditioning.2 Beginning in the late 1800s, Pavlov conducted important research on digestive secretions as well as the neural mechanisms that control them. He is, in fact, responsible for much of what we now know about digestion and won the Nobel Prize for his discoveries. Ivan P. Pavlov As part of this research enterprise, Pavlov (1849–1936) also investigated salivation, the initial step in the digestive process. By this time, he was well aware that salivation could be initiated by such psychic factors as the sight of food (today, we would use the word psychological rather than psychic). Pavlov was nevertheless surprised at the amount of control exerted by these factors. He noted, for instance, that different substances affected both the quantity and quality of saliva produced. For example, a moist, edible substance such as meat elicited a small amount of slimy saliva, whereas a dry, inedible substance such as sand elicited a large amount of watery saliva (to facilitate spitting it out). These differences existed both when the substances were actually placed in the dogs’ mouths and, later, when the dogs were merely shown these substances. Subsequent research confirmed that these psychic secretions exhibited a great deal of regularity and lawfulness, and Pavlov began to devote more and more resources to their investigation. By 1907, classical conditioning, as it would come to be known, had become the sole focus of his research efforts. In the decades that followed, Pavlov discovered most of the basic principles of classical conditioning and explored their application in such diverse areas as personality, hypnosis, sleep, and psychopathology. The epitome of the devoted scientist, Pavlov could be a tough supervisor with students and assistants (once refusing to accept an assistant’s excuse that he was late because he had had to avoid the revolutionary battles going on in the streets). Yet in other ways, he seems to have been a devoted humanitarian. When the Soviet regime took control—fortunately, it continued to support his research endeavors—he was reported to have been openly critical of its denial of basic rights and religious freedoms. Pavlov also showed great concern for the welfare of his dogs. He invested considerable effort in devising surgical procedures that allowed for the accurate observation of internal mechanisms of digestion while minimizing the animals’ discomfort and ensuring full postoperative recovery. Bettmann/CORBIS 110 2 At about the same time, an American graduate student by the name of E.!B. Twitmyer also conducted experiments on this type of conditioning, and even reported his results at the 1904 conference of the American Psychological Association (Hothersall, 1984). However, his report generated little interest, and he abandoned the topic—which is fortunate, because the term Twitmyerian conditioning is a mouthful. 3.3 Classical Conditioning Basic Procedure and Definitions We will illustrate the process of classical conditioning using one of Pavlov’s basic procedures in which a dog was trained to salivate to the sound of a metronome. During these experiments, the dog was restrained in a harness and a tube was inserted into an incision that had been made in its cheek. Whenever the dog salivated, the saliva would run down the tube into a container where it could be precisely measured (refer to Figure 3.6). Although the apparatus appears uncomfortable, the dogs in fact quickly got used to it. And the surgery involved is rather similar to a person getting a facial piercing. Pavlov’s basic procedure worked as follows. Before conditioning, the dogs automatically salivated in response to the taste of food. Because salivation to food occurs naturally (meaning that it is innate or unlearned) and does not require prior training (conditioning), it is called an unconditioned response (UR), and the food is called an unconditioned stimulus (US). The sound of a metronome, however, does not elicit salivation and is therefore said to be a neutral stimulus (NS) with respect to salivation. During conditioning, the sound of the metronome is presented just before the food, which of course continues to elicit salivation. After conditioning, as a result of having been paired with the food, the metronome itself now elicits salivation. Because salivating to the metronome requires prior training (conditioning), it is called a conditioned response (CR), and the sound of the metronome is called a conditioned stimulus (CS)3 (refer to Figure 3.7). Figure 3.6 Pavlov’s conditioning apparatus. In some of Pavlov’s early experiments, a dog was trained to salivate to the sound of a metronome. The dog was restrained in a harness, and a tube was inserted into an incision in its cheek. Whenever the dog salivated, the tube carried the saliva to a container that activated a recording device. (Source: Coon, 1998.) 3 Note that the Russian terms used by Pavlov were originally translated as conditioned and unconditioned. They are, however, more precisely translated as conditional and unconditional. In this text, however, we will continue to use the former terms because they remain more commonly used. 111 112 Chapter 3 Elicited Behaviors and Classical Conditioning Figure 3.7 Classical conditioning of salivation. Before conditioning, the dog automatically salivates to the taste of food. During conditioning, the sound of a metronome is presented just before the presentation of food. After conditioning, the metronome itself elicits salivation. (Source: Nairne, 2000.) No salivation Metronome NS Food Salivation (elicits) US UR Metronome (followed by) Food NS (elicits) US Metronome (elicits) CS Salivation UR Salivation CR This procedure can be schematically diagrammed as follows. Before conditioning: Food (US) → Salivation (UR) Metronome (NS) → No salivation (—) During conditioning: Metronome (NS): Food (US) → Salivation (UR) (Note that during this phase, the metronome is sometimes instead labeled as a CS, because that is what it eventually becomes.) 3.3 Classical Conditioning 113 After conditioning: Metronome (CS) → Salivation (CR) Each pairing of the NS and US during conditioning is called a conditioning trial.4 Several conditioning trials are often needed before the NS becomes established as a CS. In general, conditioning is more effective (requires fewer conditioning trials) when the trials are spaced far apart, such as every 10 minutes, than massed close together, such as every 30 seconds (e.g., Yin, Barnet, & Miller, 1994). Study Tip: It is important to note that the effectiveness of spaced (or distributed) learning trials over massed learning trials—which has come to be known as the spacing effect—applies to most learning situations, not just classical conditioning. For example, research has shown that studying is significantly more effective when study sessions are distributed over time (such as regular study sessions spread throughout a semester) rather than massed close together (such as cramming the night before an exam). In fact, in a comprehensive review of effective study techniques, the only techniques found to have high levels of empirical support were self-testing (as in the 3R method of studying discussed earlier) and distributed practice (Dunlosky et al., 2013). This also means that regularly reviewing older material as you proceed through this course—such as regularly testing your recall for earlier material or redoing earlier practice tests—could significantly boost your overall performance in the course. It is also important to note that simply rereading your course materials is not the best use of your time! Rereading is the most common strategy that students tend to use, but testing yourself is an efficient method for determining what you don’t know and leads to better long-term retention of information (Morehead, Dunlosky, Rawson, Blasiman, & Hollis, 2019). Measuring the level of conditioning can be done in various ways. One method is to intersperse the conditioning trials with an occasional test trial in which the NS is presented by itself. For example, every once in a while, the metronome can be presented alone to determine if it elicits salivation. Alternatively, one can continue to pair the metronome with the food and simply observe whether salivation occurs during the short interval between the start of the metronome and the presentation of food. As an everyday example of classical conditioning, let us suppose that a child is bitten by a dog and subsequently develops a fear of dogs. This process can be diagrammed as follows (omitting the “before conditioning” phase): Dog (NS): Bite (US) → Fear (UR) Dog (CS) → Fear (CR) The bite is an unconditioned stimulus that elicits an unconditioned response of fear (actually more pain than fear, but we will simplify matters 4 It is also sometimes referred to as a reinforcement trial, but in this text we will reserve the term reinforcement for operant conditioning procedures that involve the use of a “reward” (or more precisely, a “reinforcer”). Chapter 3 Elicited Behaviors and Classical Conditioning © 2021 by Sidney Harris/Science Cartoons Plus 114 Fortunately, Pavlov realized that the value of such experiments lay in their ability to reveal basic principles of behavior, not simply in their ability to make a dog salivate. a bit). As a result of the bite, the sight of the dog becomes a conditioned stimulus that elicits in the child a conditioned response of fear. Let us now look more closely at each component of the classical conditioning procedure. The unconditioned stimulus (US) is a stimulus that naturally elicits an unlearned (or innate) response, and the unconditioned response (UR) is the unlearned response that is naturally elicited by the US. For example, food naturally elicits the response of salivation, and a bite naturally elicits the response of fear (and pain). (Note that the US and UR are sometimes given the abbreviations of UCS and UCR.) The conditioned stimulus (CS) is any stimulus that, although initially neutral, comes to elicit a learned response because it has been associated (or paired) with an unconditioned stimulus. The metronome is initially neutral with respect to salivation in that it does not naturally elicit salivation.5 After the metronome has been associated with food, however, it does elicit salivation. The conditioned response (CR) is the learned response, often similar to the UR, that is elicited by the CS. Note that the conditioned response is at most only similar to the unconditioned response. It is never identical (a fact that is overlooked in many introductory psychology textbooks). Even when the UR and CR appear identical, as in the case of salivation elicited by the food (US) and by the metronome (CS), there are always some differences. For example, the CR is usually weaker or less intense 5 Although the sound of the metronome is neutral with respect to salivation, it might not be neutral with respect to other types of responses. For example, the sound will very likely function as a US for an orienting response (turn on the metronome and the dog will naturally prick up its ears and turn toward it). 3.3 Classical Conditioning 115 than the UR. Thus, the dog will typically salivate less to the metronome than it will to the food. The CR is also sometimes quite different from the UR. For example, as noted earlier, the unconditioned response elicited by a dog bite may be very different from the conditioned response elicited by the sight of the dog that bit us. For simplicity, we labeled both responses as fear responses; however, the response to the bite is most likely what we would describe as a pain reaction (“Yeow!”), whereas the subsequent response to the sight of the dog would be one that is more clearly identified as fear (e.g., freezing). The extent to which the CR can differ from the UR is discussed more fully in Chapter 5. 2. 3. 4. 5. 6. conditioning conditioning. In the latter case, or what Skinner called r behaviors or simply the behaviors themselves are called r . In the metronome example, the metronome is initially a(n) stimulus because it (does/does not) elicit salivation. The food, however, is a(n) stimulus that naturally elicits a(n) response of salivation. Following conditioning, the metronome is a(n) stimulus, and the salivation elicited by the metronome is a(n) response. Each pairing of the metronome and the food is called a c tr ; conditioning is most efficient, in the sense of fewer conditioning trials being needed, when these pairings are spaced (far apart/close together). In the basic classical conditioning procedure, the (US/NS) is followed by the (CS/ US/NS), which in turn elicits the (CR/UR). As a result, the first stimulus becomes a (CS/US/NS), which elicits a (CR/UR). Using the appropriate abbreviations, label each component in the following diagram of a conditioning procedure: Wasp ( ): Painful sting ( Wasp ( ) → Fear ( ) → Fear ( ) ) 7. Using the format in question 6, diagram a classical conditioning procedure involving the stimuli of “nurse” and “painful injection,” and the response of “anxiety.” Label each component using the appropriate abbreviations. 8. The UR is the (learned/innate) response to the (NS/CS/US). By contrast, the CR is response to the . the 9. The CS in a conditioning procedure was originally a(n) (NS/US/UR). The CR in a conditioning procedure is (often/always) (similar/identical) to the UR. Appetitive and Aversive Conditioning Most classical conditioning procedures can be divided into two categories based on whether the US is pleasant or unpleasant. In appetitive conditioning, the US is an appetitive event (one that an organism approaches or seeks out); in mentalistic terms, it is something usually regarded as pleasant. Examples include food (if the organism is hungry), water (if the organism is thirsty), or addictive drugs. Sexual stimuli are also regarded as appetitive stimuli, and there is good evidence that sexual Quick Quiz G 1. Classical conditioning is also known as P 116 Chapter 3 Elicited Behaviors and Classical Conditioning responses can be classically conditioned. For example, Rachman and Hodgson (1968) took seven male volunteers and presented them with conditioning trials in which a picture of black, knee-length boots was followed by a picture of a nude woman. After about 30 trials, five of the males became sexually aroused by the sight of the boots. (Don’t worry. The researchers later eliminated the conditioning by repeatedly presenting the picture of the boots without the picture of the nude—a process known as extinction that we later discuss.) Interestingly, decades later, other researchers wondered whether an even more neutral stimulus (one that isn’t generally associated with the human body) could also become a CS for sexual arousal. They indeed found that conditioning occurred (after a mere 15 trials) when a jar of pennies was used as the CS (Plaud & Martini, 1999)—giving a whole new meaning to the term love of money! In aversive conditioning, the US is an aversive event (one that an organism avoids); that is, in mentalistic terms, it is something we usually regard as unpleasant. Examples of aversive USs include an electric shock, a painful bite, and an unpleasant odor. Aversive conditioning often occurs rapidly, perhaps with only one or two conditioning trials, especially when the aversive stimulus is quite strong. This reflects the close relationship between aversive conditioning and survival; to survive, we have evolved in such a way as to quickly learn to dislike those events that cause pain or illness. Given how easily aversive conditioning can occur, it is not surprising that this type of conditioning helps account for many of our fears and anxieties. When the fear is appropriate—as in learning to fear a vicious dog that has bitten us— such conditioning is beneficial. When the fear is inappropriate—as when we begin to fear all dogs—such conditioning can be problematic. Therefore, a great deal of effort has gone into the study of fear conditioning, as well as into how such fears can be eliminated. This research has yielded important information on how real-world fears and anxieties can be treated (and is discussed more fully in Chapter 5). When conducting research on fear conditioning in animals, measuring the level of fear can be problematic. Changes in certain physiological responses, such as heart rate, that might indicate fear are difficult to record, especially in small experimental animals such as rats. An ingenious solution to this problem was developed by Estes and Skinner (1941); it is known as the conditioned suppression or conditioned emotional response (CER) paradigm. In the basic paradigm, the rat is first trained to press a lever to obtain food (with many lever presses being required to obtain a single pellet). When a steady rate of lever pressing has been established, a fear-conditioning procedure is introduced in which, say, a 30-second tone is presented followed by a 1-second shock. Thus: 30" Tone (NS): 1" Shock (US) → Fear (UR) In the initial phase, the rat will become emotionally upset (fearful) whenever it receives a shock and will stop pressing the lever for food. As conditioning proceeds, however, the tone also will come to elicit fear, 30" Tone (CS) → Fear (CR) 3.3 Classical Conditioning and the rat will stop pressing the lever when it hears the tone. Thus, the degree to which lever pressing for food is suppressed in the presence of the 30-second tone can be used as an indirect measure of the extent to which the tone elicits fear. Think of the procedure as similar to a scene from an old western movie, where a gunfighter walks into a saloon. The extent to which the saloon patrons fear the gunfighter can be accurately measured by the extent to which they stop talking to each other (or drinking beer) when he is in the saloon (you can hear a pin drop!) and then resume talking (or drinking) when he leaves. Similarly, the rat’s level of fear can be assessed by the extent to which it stops lever pressing for food when the tone is sounding and resumes lever pressing for food when the tone is not sounding. On a more formal level, conditioned suppression is measured in the form of a suppression ratio. A suppression ratio is the number of responses emitted during the CS period (while the tone is on) divided by the total number emitted during the CS period and the same length period immediately preceding the CS. Thus, Suppression Ratio 5 # of CS responses # of CS responses 1 # of pre‒CS responses For example, imagine that a rat emits 20 responses during a 30-second pre-CS period followed by 0 responses during the 30-second CS period. In other words, there is total suppression of responding during the CS period. The suppression ratio would be: 0 50 0 1 20 Thus, a suppression ratio of 0 indicates total suppression of responding. But what if instead there were only a partial suppression of responding during the CS? For example, what if the rat emitted 10 responses during the CS period? In this case, the suppression ratio would be: 10 10 5 5 .33 10 1 20 30 And if there were no suppression of responding—that is, the rat emitted the same number of responses during the CS period as during the pre-CS period—the suppression ratio would be: 20 20 5 5 .5 20 1 20 40 Note how the suppression ratio will generally vary between 0 and .5, with a lower ratio indicating greater suppression and more effective conditioning. A ratio of 0 therefore indicates greater suppression and stronger fear conditioning than a ratio of .33, which in turn indicates greater suppression and stronger fear 117 118 Chapter 3 Elicited Behaviors and Classical Conditioning Quick Quiz H conditioning than a ratio of .5. Students often find this confusing, as stronger conditioning is indicated by the lower number, which is opposite to the way most ratios work. To keep it straight, simply remember that a lower number indicates less responding (greater suppression) of behavior. The CER paradigm has proven to be a useful method for investigating fear conditioning in animals and is, in fact, commonly used to study classical conditioning processes. But be careful to remember that the CR in this type of procedure is assumed to be the covert response of fear; it is not the reduction in lever pressing, which simply serves as the indirect measure of fear. Note that classical conditioning can also transform a normally aversive stimulus into an appetitive stimulus. Pavlov found that if a dog received a shock to one of its paws and then received food, the dog would eventually begin to salivate in response to the shock. The dog’s overt reactions to the shock, such as tail wagging, further indicated that the shock had lost its aversiveness. Interestingly, if the shock were then applied to a different paw, the dog would not salivate but would instead react with discomfort. The perception of shock as pleasurable appeared to be quite specific to the body part involved in the conditioning. As you may already have guessed, this same process might partially account for the development of masochistic tendencies (the tendency to perceive painful stimulation as pleasurable) in humans. The painful stimulation from being whipped, for example, has for some people become associated with feelings of sexual arousal; as a result, the painful stimulation itself can now elicit arousal. Interestingly, as with Pavlov’s dogs, people who are masochistic do not perceive all pain as pleasurable; rather, it is only the type of pain that is connected with their erotic experiences (e.g., being whipped) that is perceived as pleasurable. The pain they feel from accidentally stubbing a toe or banging a shin is typically as aversive for them as it is for anyone else (Rathus, Nevid, & Fichner-Rathus, 2000). 1. In 2. 3. 4. 5. 6. conditioning, the US is an event that the organism conditioning, the US is an event typically avoids; in that the organism typically seeks out. Learning to associate the neighborhood café with the happy times you experience in that café is an example of (aversive/appetitive) conditioning; learning to associate your refrigerator with the nauseating smell of spoiled food is an example of (aversive/appetitive) conditioning. In a c e response (CER) paradigm, the level of fear elicited by a CS is indicated by the degree to which the rat’s rate of lever pressing for food (decreases/increases) in the presence of that stimulus. The CER paradigm is also known as a c s procedure. The suppression ratio is the number of (pre-CS/CS/post-CS) responses divided by the responses plus responses. number of Intense fear in a CER procedure is indicated by a suppression ratio of (.5/0); no fear is indicated by a suppression ratio of about (.5/0). 3.3 Classical Conditioning 119 And Furthermore Classical Conditioning and Interpersonal Attraction Classical conditioning may play an important role in interpersonal attraction. According to the reinforcement–affect model of attraction (Byrne & Clore, 1970), the extent to which we are attracted to someone can be significantly affected by the degree to which the person is associated with events that elicit positive emotions. For this reason, we are generally attracted to people who say and do the kinds of things that make us feel good. Even events as innocuous as pleasant background music or a positive news story on the radio can heighten our perception of someone as attractive. This means that associating ourselves with pleasant stimuli—pleasant music, attractive clothing, and even a clean car—during an initial date can greatly facilitate the possibility of a second date. The reinforcement–affect model also suggests that we are less attracted to someone who is associated with aversive events. Obviously, dressing like a slob or drinking to the point of vomiting during a first date is probably not a good idea. But hearing bad news on the radio or really annoying music could also undermine your prospects for a second date. On the other hand, there may be times when you want to be perceived as less attractive. A letter once appeared in a newspaper advice column in which a woman described how she finally managed to dissuade a persistent acquaintance from continually asking her out. She agreed to a date and then ate plenty of garlic beforehand! Her suitor was apparently not a big garlic fan, and she had no further problems with him. Of course, none of this would be necessary if people learned to take “no” for an answer (which will hopefully be one result of the “Me Too” movement). Excitatory and Inhibitory Conditioning In all of the examples so far, and as it is traditionally defined, the NS is associated with the presentation of a US. The metronome is associated with the presentation of food, the dog is associated with a painful bite, and the tone is associated with shock. Conditioning in which a stimulus is associated with the presentation of a US and therefore (as a CS) comes to elicit a certain response is known as excitatory conditioning. But what if a stimulus is associated with the absence of the US rather than its presentation? What if, for example, a vicious dog always bites you except when its owner is present? The owner then is a sort of safety signal that indicates the absence of a painful bite. Conditioning in which a stimulus is associated with the absence or removal of a US and therefore (as a CS) inhibits a certain response is known as inhibitory conditioning. Therefore, the result of inhibitory conditioning is that a certain response is less likely to occur when the CS is present. Thus, although the dog is an excitatory CS for fear, the owner is an inhibitory CS for fear, and your fear of the dog will be inhibited when the owner is present. Similarly, if a rat is consistently shocked after a tone is presented, the tone will become an excitatory stimulus for fear. But if 120 Chapter 3 Elicited Behaviors and Classical Conditioning Quick Quiz I the rat is never shocked after a tone and a light are presented together, the light will become an inhibitory CS for fear because it signals the absence of shock. In such procedures, the excitatory CS is usually labeled a CS+, and the inhibitory CS is labeled a CS–. Traditionally, researchers have focused on the study of excitatory conditioning, and most of the basic principles of classical conditioning have been established using excitatory procedures. For this reason, most of the examples in this text are examples of excitatory conditioning. In recent years, however, the study of inhibitory conditioning has begun to attract a good deal of attention (Domjan, 2015). 1. Conditioning trials in which a stimulus is followed by the presentation of a US conditioning in which the CS comes to a response ; conditioning trials in which a stimulus is followed by the absence or removal of a US will produce a response. conditioning in which the CS comes to Your grandmother always cooks great meals except when your picky sister is present. As a result, you usually salivate a lot when sitting at your grandmother’s table for a meal but not when your sister is present. Your grandmother’s table CS for salivation, while your picky sister is a(n) is a(n) CS for salivation. Most of the basic principles of classical conditioning have been established using conditioning. procedures that involve An excitatory CS is one that is associated with the (presentation/removal) of a US; an inhibitory CS is one that is associated with the (presentation/removal) of a US. An excitatory CS for fear is one that will (elicit/suppress) a fear response; an inhibitory CS for fear is one that will (elicit/suppress) a fear response. For the residents of Berlin and London during World War II, an air-raid siren would have been a (CS+/CS–) for anxiety, while the all-clear siren would have been a (CS+/CS–) for anxiety. A click is followed by food, while a click and a buzzing noise is never followed by food. In this case, the click will become a (CS+/CS–) for salivation and the buzzing noise will become a (CS+/CS–). will produce 2. 3. 4. 5. 6. 7. Temporal Factors in Conditioning In the classical conditioning examples discussed to this point, the NS was always presented before the US. This arrangement, however, is only one of several ways to present the NS and US. In this section, we outline four NS– US arrangements and note their effectiveness in producing conditioning. We then consider a form of conditioning, temporal conditioning, in which the NS (CS) is not an external stimulus but is itself the passage of time. Four Temporal NS–US Arrangements There are essentially four ways in which presentations of the NS and US can be temporally arranged. We outline these arrangements next and note the effectiveness of each for producing a conditioned response. 3.3 Classical Conditioning 1. Delayed Conditioning. In delayed conditioning, the onset of the NS precedes the onset of the US, and the two stimuli overlap. For example, if we want a rat to associate a tone with a brief shock, we first present the tone and then, while the tone is still on, present a shock. As shown in Figure 3.8a, the onset of the tone precedes the onset of the shock and the tone is still on when the shock is presented. (Note that it is the point at which the two stimuli are turned on, rather than turned off, that is critical.) A delayed conditioning procedure is often the best arrangement for conditioning, especially when the time between the onset of the NS and the onset of the US (known as the interstimulus interval or ISI) is relatively brief. When conditioning certain autonomic responses (responses controlled by the autonomic nervous system), such as salivation, the optimal ISI is often in the range of a few seconds. When conditioning skeletal responses (responses controlled by skeletal muscles), such as the eyeblink reflex, the optimal ISI is about a half second. Thus, conditioning generally works best when the onset of the NS is soon followed by the onset of the US; this fact is consistent with the idea that the NS serves as a predictor of the US, a notion that is discussed in Chapter 5. (Nevertheless, some types of classical conditioning do not require a close temporal pairing between the NS and US. One such type, known as taste aversion conditioning, is described in Chapter 12.) 2. Trace Conditioning. In trace conditioning, the onset and offset of the NS precede the onset of the US. In other words, the NS occurs before the US, and the two stimuli do not overlap. For example, a tone is turned on and off and is then followed by the presentation of a shock (refer to Figure 3.8b). The time between the offset of the NS and the onset of the US (e.g., between the point when the tone was turned off and the shock was turned on) is called the trace interval. Because the tone is no longer present when the shock occurs, you might say that the organism has to “remember” the occurrence of the tone (or, in cognitive terms, have a “memory trace” of it) to be able to associate the two. Trace conditioning can be almost as effective as delayed conditioning if the trace interval is relatively short (no more than a few seconds). If the trace interval is longer than that, conditioning is, in most cases, much less likely to occur. 3. Simultaneous Conditioning. In simultaneous conditioning, the onset of the NS and the onset of the US occur simultaneously. For example, a tone and a shock are turned on at the same time (refer to Figure 3.8c). Although simultaneous conditioning involves the closest possible contiguity between the NS and the US, this procedure usually results in poor conditioning. One reason for this is that if the NS occurs at the same time as the US, the NS is no longer a good predictor of the US (in the same way, for example, that a “check engine” light on the dashboard of your car isn’t very useful if it starts flashing at precisely the same time as the engine stops working; it’s useful only if it warns you ahead of time that the engine is about to fail.). 4. Backward Conditioning. In backward conditioning, the onset of the NS follows the onset of the US. In other words, the US is presented first and the NS is presented later. For example, the rat receives a shock and then hears 121 122 Chapter 3 Elicited Behaviors and Classical Conditioning Four ways in which presentation of the NS and US can be temporally arranged. Figure 3.8 (a) Delayed conditioning procedure (onset) (offset) Tone (NS) Shock (US) (onset) (offset) Time (b) Trace conditioning procedure Tone (NS) Shock (US) Time (c) Simultaneous conditioning procedure Tone (NS) Shock (US) Time (d) Backward conditioning procedure Tone (NS) Shock (US) Time a tone (refer to Figure 3.8d). Backward conditioning has traditionally been regarded as the least effective procedure for conditioning, especially with respect to conditioning of an excitatory response (although, as you will find in Chapter 12, exceptions can occur, such as with stimuli that we may have an innate tendency to fear). Backward conditioning can, however, sometimes result in conditioning of an inhibitory response. For example, if a tone sounds just before a shock is terminated, then the tone reliably predicts the removal of shock. The tone in this case can become a safety signal (which can be given the symbol CS-) that inhibits the occurrence of fear. Thus, although delayed conditioning is traditionally thought of as the most effective arrangement, conditioning can occur with other arrangements. In fact, although beyond the scope of this text, recent evidence suggests that all of these arrangements can, in complex ways, somehow impact behavior (refer to Domjan, 2015). 3.3 Classical Conditioning 123 Temporal Conditioning In all of the examples presented to this point, the CS has been a distinctive external stimulus of some sort, such as a light, a metronome, or a dog. But this need not be the case. Temporal conditioning is a form of classical conditioning in which the CS is the passage of time. For example, if a dog is given a bite of food every 10 minutes, it will eventually salivate more strongly toward the end of each 10-minute interval than at the start of the interval. The end of the 10-minute interval is the effective CS for salivation. Similarly, residents of a city who experience a bombing attack each night at 2:00 ".#. for several nights in a row will likely start feeling anxious as 2:00 ".#. approaches, even in the absence of any clock indicating the time. The internal cues that one uses to estimate time are sufficient to elicit the feelings of anxiety. 2. 3. 4. 5. 6. 7. 8. conditioning, in which the onset of the NS (precedes/follows) the onset of the US, and the two stimuli (overlap/do not overlap). In delayed conditioning, the time between the onset of the NS and the onset of interval (abbreviated ). the US is called the In trace conditioning, the (onset/offset/onset and offset) of the NS precedes the of the US. In trace conditioning, the time between the of the NS of the US is called the and the interval. Trace conditioning can be effective if this interval is relatively (long/short). In simultaneous conditioning, the of the NS occurs at of the US. the same time as the In backward conditioning, the (NS/US) is presented first and the (NS/US) is presented later. Backward conditioning can result in inhibitory conditioning when the NS signals the (presentation/removal) of the US. Simultaneous and backward conditioning procedures have traditionally been considered as (less/more) effective than delayed and trace conditioning procedures. Suppose that we attempt to condition a reflex response of sneezing using a flower as the NS and pollen as the US. Name each of the four temporal arrangements listed. a. conditioning: Flower | | Pollen b. | | conditioning: Flower | | Pollen | | c. conditioning: Flower | | d. conditioning: Flower | Pollen | | | Pollen | | conditioning, the (NS/US) is presented at regular intervals, with the result that the passage of time becomes a (CS/US). 9. In Quick Quiz J 1. The most effective temporal arrangement for conditioning is 124 Chapter 3 Elicited Behaviors and Classical Conditioning Pseudoconditioning We hope you are now pretty familiar with the basic classical conditioning procedure and some of the phenomena associated with it. Be aware, however, that determining whether classical conditioning has occurred is not always as straightforward as it might seem. A phenomenon known as pseudoconditioning poses a particular problem. In pseudoconditioning, an elicited response that appears to be a CR is actually the result of sensitization rather than conditioning. Suppose, for example, that we try to condition a leg withdrawal reflex (leg!flexion) in a dog by presenting a light flash followed by a slight shock to!its paw. Light flash: Shock → Leg flexion After a few pairings of the light with the shock, we now find that a fairly strong flexion response occurs immediately when the light is flashed. Light flash → Leg flexion On the surface, it seems that the light flash has become a CS and that we have successfully conditioned a flexion response. But have we? What if instead of flashing a light, we sound a beep and find that, lo and behold, it too elicits a response? Beep → Leg flexion What is going on here? Remember the process of sensitization, in which the repeated presentation of an eliciting stimulus can sometimes increase the strength of the elicited response? Well, sensitization can result in the response also being elicited!by other stimuli. For example, soldiers with war trauma exhibit an enhanced startle response, not just to the sound of exploding mortar shells but to certain other stimuli as well, including doors slamming, cars backfiring, or even an unexpected tap on the shoulder. Similarly, if a dog has been shocked in the paw a couple of times, it would not be at all surprising if any sudden stimulus in that setting could make the dog quickly jerk its leg up. Therefore, although we thought we had established a CR—which is the result of a CS having been paired with a US such that it now predicts the occurrence of the US—in reality, we have simply produced a hypersensitive dog that automatically reacts to almost any sudden stimulus, even one that has never been paired with the US and doesn’t predict its occurrence. Thus, what appears on the surface to be an example of classical conditioning is actually a false example of conditioning (the word pseudo means “false”). Pseudoconditioning is a potential problem whenever the US is some type of emotionally arousing stimulus. Fortunately, it is possible to assess the extent to which a response is the result of pseudoconditioning rather than real conditioning. One way to do this is to employ a control condition in which the NS and US are presented separately. For example, subjects in the experimental group receive several pairings of the light flash and the shock, In Review 125 while subjects in the control group receive light flashes and shocks that are well separated in time (which we will indicate with a double slash: / /). Experimental group Control group Light flash: Shock → Leg flexion Light flash / / Shock → Leg flexion When the animals in each group are later exposed to the light flash presented on its own, we find the following: Experimental group Control group Light flash → Strong leg flexion Light flash → Weak leg flexion The level of responding shown by the control group is presumed to reflect the amount of sensitization (pseudoconditioning) resulting from the use of an upsetting stimulus (the shock) as the US. However, because the response shown by the experimental group is stronger than that shown by the control group, conditioning is assumed to have occurred, with the difference between the two groups indicating the strength of conditioning. More specifically, the strong leg flexion in the experimental group is the result of sensitization plus conditioning, while the weak leg flexion in the control group is the result of sensitization alone. The difference between the two groups therefore indicates the amount of conditioning that occurred in the experimental group. Classical conditioning experiments typically use one or more control groups like this to assess how much actual conditioning has taken place versus the extent to which the subject’s responses are the result of nonconditioning factors such as sensitization. sensitization, we say that has taken place. 2. The phenomenon in the previous question is a potential problem whenever the US produces a strong em response. 3. An appropriate control procedure to test for this phenomenon involves a control group of subjects being presented the NS and US (together/apart). Responding that is later elicited by the NS in this group is assumed to be the rather than real conditioning. result of s In Review To create a summary for this chapter, include a paragraph about elicited behaviors and another paragraph about how and why those elicited behaviors can change in response to exposure. You’ll likely need at least two paragraphs to summarize the mechanisms and types of classical conditioning. Other ways to test yourself on concepts from this chapter include creating simple diagrams. Start by replicating diagrams associated with Pavlov’s experiments (in which food is the US that elicits the UR of salivation) and then create additional diagrams associated with other examples in the text. You should also be able to use the formula to calculate the strength of a CER. Finally, Quick Quiz K 1. When an elicited response that appears to be a CR is actually the result of 126 Chapter 3 Elicited Behaviors and Classical Conditioning try creating your own examples of conditioning from things you’ve observed or experienced in your life, including both negative events and positive events. Further Exploration Habituation is a real problem for people and organizations focused on wildlife conservation. Review this statement about habituation from the U.S. National Park Service: https://www.nps.gov/grca/learn/nature/wildlife-habituation.htm. Do you have an allergy? You might be interested in this discussion from the British Society for Immunology of how the process of sensitization is important for the development of allergies: www.immunology.org/policy-and -public-affairs/briefings-and-position-statements/allergy. Most people think about operant conditioning for animal training (and we will cover that soon!) but classical conditioning can be equally important. Here’s a great example of how horse training is partly dependent on classical conditioning: www.equinebehaviorist.ca/post /classical-conditioning-and-horses. Study Questions 1. What is a reflex? Diagram the sequence of events in a reflex arc that underlies a flexion response when one’s hand touches a sharp object. 2. Define fixed action pattern. What is a sign stimulus or releaser? Give an example of a fixed action pattern and its releaser. 3. Define habituation and sensitization. What is the effect of stimulus intensity on habituation and sensitization? 4. Distinguish between long-term and short-term habituation. Describe the phenomenon of dishabituation. 5. Define the opponent-process theory of emotion. What are four characteristics of opponent processes? 6. Define classical conditioning. What other terms are used for this type of conditioning? 7. Diagram an example of a classical conditioning procedure. Be sure to label each component using the appropriate abbreviations. 8. Define the terms unconditioned stimulus, unconditioned response, conditioned stimulus, and conditioned response. 9. Distinguish between appetitive and aversive conditioning. Describe the conditioned suppression (or CER) procedure. 10. How do excitatory conditioning and inhibitory conditioning differ from each other? Give an example of each. 11. Name and diagram (or describe) four temporal arrangements of the NS and US. Which two arrangements are traditionally considered to be most effective? What is temporal conditioning? 12. What is pseudoconditioning and how is it assessed? Concept Review Concept Review appetitive conditioning. Conditioning procedure in which the US is an appetitive event (one that an organism approaches or seeks out). aversive conditioning. Conditioning procedure in which the US is an aversive event (one that an organism avoids). backward conditioning. Conditioning procedure in which the onset of the NS follows the onset of the US. classical conditioning. A process whereby a stimulus comes to elicit a response because it has been paired with (or associated with) another stimulus. Also known as Pavlovian conditioning or respondent conditioning. conditioned response (CR). The response, often similar to the unconditioned response, that is elicited by the conditioned stimulus. conditioned stimulus (CS). Any stimulus that, although initially neutral, comes to elicit a response because it has been associated with (paired with) an unconditioned stimulus. delayed conditioning. Conditioning procedure in which the onset of the NS precedes the onset of the US, and the two stimuli overlap. dishabituation. The reappearance (or recovery) of a habituated response to a stimulus following the presentation of another, seemingly irrelevant novel stimulus. elicited behavior. Behavior that is drawn out (elicited) by a preceding stimulus. Also known as respondent behavior. excitatory conditioning. Conditioning procedure in which a stimulus is associated with the presentation of a US and therefore, as a CS, comes to elicit a response. fixed action pattern. A fixed sequence of responses elicited by a specific stimulus. flexion response. The automatic response of jerking one’s hand or foot away from a hot or sharp object. habituation. A decrease in the strength of an elicited response following repeated presentations of a stimulus that naturally (without prior learning) elicits that response. inhibitory conditioning. Conditioning procedure in which a stimulus is associated with the absence or removal of a US and therefore, as a CS, comes to inhibit the occurrence a response (meaning that the response is less likely to occur when the CS is present than when it is absent). opponent-process theory. A theory proposing that an emotional event elicits two competing processes: (1) an a-process (or primary process) that is directly elicited by the event, and (2) a b-process (or opponent process) that is elicited by the a-process and serves to counteract the a-process. orienting response. The automatic positioning of oneself to facilitate attending to a stimulus. pseudoconditioning. False conditioning in which an elicited response that appears to be a CR is actually the result of sensitization rather than conditioning. 127 128 Chapter 3 Elicited Behaviors and Classical Conditioning reflex. A relatively simple, involuntary response to a stimulus. reflex arc. A neural structure that underlies many simple reflexes and consists of a sensory neuron, an interneuron, and a motor neuron, in that order. sensitization. An increase in the strength of an elicited response following repeated presentations of the stimulus that naturally (without prior learning) elicits that response. sign stimulus (or releaser). A specific stimulus that elicits a fixed action pattern. simultaneous conditioning. Conditioning procedure in which the onset of the NS and the onset of the US are simultaneous. startle response. A defensive reaction to a sudden, unexpected stimulus, which involves the automatic tightening of skeletal muscles and various hormonal and visceral changes. temporal conditioning. A form of classical conditioning in which the CS is the passage of time. trace conditioning. Conditioning procedure in which the onset and offset of the NS precede the onset of the US. unconditioned response (UR). The response that is naturally (without prior learning) elicited by the unconditioned stimulus. unconditioned stimulus (US). A stimulus that naturally (without prior learning) elicits a response. Chapter Test 4. A sudden loud noise is likely to elicit a(n) response, which is a reflexive defensive response to a sudden, unexpected stimulus. 13. With repeated presentations of the emotional event, the b-process (increases/ decreases) in both ____________________ and ____________________. 23. Seeing an insect on your arm and then watching it bite you is an example of a(n) conditioning procedure; noticing the insect at the same moment that you feel the bite is an example of a(n) conditioning procedure. The (former/latter) scenario is likely to result in a stronger fear of the insect. 6. When a subordinate dog submits to a threatening display from a dominant dog, it will often roll over on its back and display its belly. This type of action sequence is called a(n) , and the threatening display from the dominant dog is called the stimulus or for these actions. 14. Classical conditioning is also known as P conditioning or conditioning. In the latter case, the elicited behaviors are referred to as . Chapter Test 9. The faint sound of a jackhammer several blocks away will likely result in ____________________, but the extremely loud sound of a jackhammer right outside your window might result in ____________________. The moderately loud sound of a jackhammer half a block away might result in a period of ____________________ followed by ____________________. 26. In general, aversive conditioning occurs (more/less) readily than appetitive conditioning. 12. According to the opponent-process theory of emotion, b-processes are (slow/quick) to increase and (slow/quick) to decrease. 18. Imagine an eyeblink conditioning procedure in which the sound of a click is paired with a puff of air to the eye. Each pairing of the click and air puff during conditioning is referred to as a(n) ____________________. 2. During an eyeblink conditioning procedure, you blinked not only in response to the sound of the click but also when someone tapped you on the shoulder. Your response to the tap on the shoulder may be indicative of _________conditioning, which means that the elicited eyeblink may be indicative of ____________________ rather than classical conditioning. 11. In the opening scenario to this chapter, Uma witnessed her boyfriend flirting with another woman. First, she experienced intense anger. Later, however, when he apologized for his actions and was very attentive to her, she experienced unusually strong feelings of attraction toward him. An explanation for this pattern of emotional changes is provided by the theory of emotion. In this case, Uma’s feelings of anger are an example of the process, and her feelings of affection following his apology are an example of the ____________________ process. 1. Behaviors that are automatically drawn out by the stimuli that precede them are referred to as behaviors (or what Skinner called ____________________ behaviors). 20. When you opened your refrigerator yesterday, the putrid smell of rotten eggs made you nauseous. Today, when you are about to open the refrigerator, you again find yourself becoming nauseous, even though the refrigerator has been thoroughly cleaned. In classical conditioning terms, the refrigerator has become a(n) stimulus that now elicits a(n) ____________________ response of nausea. In this case, the nausea produced by the sight of the refrigerator is likely to be (less/ more) severe than the nausea produced by the smell of rotten eggs. 5. The reflexive action of pulling your hand away from a hot plate is activated through a(n) ____________________: a neural structure underlying simple reflexes that consists of (in correct order) ___________________________ ______________ ______________________________________________. 25. A conditioning procedure is likely to be more effective if the conditioning trials are spaced (one minute/one hour) apart. This is in keeping with what is known as the effect, in which learning occurs more readily when practice sessions are spaced (close together/far apart). 129 130 Chapter 3 Elicited Behaviors and Classical Conditioning 10. You finally habituate to the faint sound of a jackhammer half a block away, such that you cease to notice it. The lights in your house then go out, at which point you again notice the sound of the jackhammer. This is an example of the process of ____________________. 15. Imagine an eyeblink conditioning procedure in which the sound of a click is paired with a puff of air to the eye. The puff of air is called the stimulus (abbreviated ), and the eyeblink that it elicits is called the response (abbreviated ). 30. In general, long-term habituation is most likely to occur when the stimulus is presented at (narrowly/widely) spaced intervals and your ability to respond to the stimulus will recover if that stimulus has been absent for a (long/short) period of time. 19. When you opened your refrigerator one evening, the putrid smell of rotten eggs made you nauseous. In classical conditioning terms, the putrid smell is a(n) stimulus that elicits a(n) ____________________ response of nausea. 28. Inadvertently touching a hot object is likely to elicit a(n) _________________ response, which is controlled through a simple sequence of neurons known as a ____________________. 7. Fixed action patterns are sometimes called behaviors because they are often unique to a certain species. 3. The reflexive action of a dog pricking up its ears in response to a sound is an example of a(n) response, which consists of movements designed to facilitate . 32. How does one calculate a suppression ratio? 17. Imagine an eyeblink conditioning procedure in which the sound of a click is paired with a puff of air to the eye. After conditioning, the click becomes a(n) stimulus (abbreviated ) because it now elicits an eyeblink. The eyeblink elicited by the click is called the response (abbreviated ). 27. Dana always feels relaxed when she takes her large dog for a walk, even though the neighborhood is relatively dangerous. This appears to be an example of , with the dog functioning as a(n) ____________________ CS (which can be abbreviated ____________________). 21. When you opened the refrigerator one evening, the putrid smell of rotten eggs made you nauseous. Your later response of nausea to the sight of the refrigerator is an example of (aversive/appetitive) conditioning as well as (excitatory/inhibitory) conditioning. 31. Jared’s parents always start arguing at around midnight each night. As a result, he wakes up feeling anxious each night just before midnight. This seems to be an example of ____________________ conditioning. Answers to Chapter Test 16. Imagine an eyeblink conditioning procedure in which the sound of a click is paired with a puff of air to the eye. Before conditioning, the sound of the click does not elicit an eyeblink; it is therefore considered a(n) stimulus. 29. In a conditioned suppression ratio, a score of indicates total suppression of the behavior, while a score of indicates no suppression. 8. In a restaurant, the parents of a noisy child hardly notice the commotion. This is an example of . However, the customers at nearby tables are becoming increasingly annoyed by the child. This is an example of . 22. Brett is allergic to bee stings. He eats and drinks heartily when he is inside a restaurant, but not when he is seated on the outdoor patio surrounded by flowers. This circumstance is similar to the paradigm, which is also known as the (CER) procedure. 24. In an experiment involving the conditioning of an eyeblink response to the sound of a click, hearing the click and then two seconds later feeling the puff of air in your eye is an example of a(n) conditioning procedure. Conversely, feeling the puff of air and then hearing the click is an example of a(n) conditioning procedure. In general, the (former/latter) procedure is likely to be more effective. Answers to Chapter Test 1. elicited; respondent 2. pseudo; sensitization 3. orienting; attending to a stimulus 4. startle 5. reflex arc; sensory neuron; interneuron; motor neuron 6. fixed action pattern; sign; releaser 7. species-specific 8. habituation (or long-term habituation); sensitization 9. habituation; sensitization; sensitization; habituation 10. dishabituation 11. opponent-process; primary (or a-); opponent (or b-) 12. slow; slow 13. increases; strength; duration 14. Pavlovian; respondent; respondents 15. unconditioned; US; unconditioned; UR 16. neutral 17. conditioned; CS; conditioned; CR 18. conditioning trial 19. unconditioned; unconditioned 20. conditioned; conditioned; less 21. aversive; excitatory 22. conditioned suppression; conditioned emotional response 23. delayed; simultaneous; former 24. trace; backward; former 25. one hour; spacing; far apart 26. more 27. inhibitory conditioning; inhibitory; CS– 28. flexion; reflex arc 29. 0; .5 30. widely; long 31. temporal 32. Assuming time periods of equal length, divide the number of CS responses by the number of CS responses plus pre-CS responses 131 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Jana enjoys being wildly unpredictable in her relationships, believing that most men find unpredictable women exciting. She cancels dates at the last minute, shows up on the guy’s doorstep at odd hours of the day or night, and tries to be completely spontaneous as often as possible. Unfortunately, many of the guys she goes out with seem to be rather stressed out and neurotic, though it usually takes a while before this becomes apparent. She is starting to wonder if there are any good men around these days. Chapter Outline 4.1 Some Basic Conditioning Phenomena Acquisition Extinction and Related Phenomena Stimulus Generalization and Discrimination Experimental Neurosis 4.2 Three Extensions to Classical Conditioning Sensory Preconditioning US Revaluation 4.3 Four Examples of Specificity in Classical Conditioning Overshadowing Blocking Occasion Setting Latent Inhibition Higher-Order Conditioning Learning Objectives 4-1 Describe learning acquisition and extinction, including response patterns associated with each. 4-2 Identify factors that influence the likelihood of acquisition or extinction. 132 4.1 Some Basic Conditioning Phenomena 4-3 Distinguish between generalization and discrimination, including factors that influence which is more likely to occur. 4-4 Define and distinguish among the extensions of classical conditioning. 4-5 Define and distinguish among the examples of specificity in classical conditioning. 4-6 Using examples from this chapter, describe how context and prior learning influence current learning situations. 4.1 Some Basic Conditioning Phenomena Acquisition In classical conditioning, acquisition refers to the process of developing and strengthening a conditioned response through repeated pairings of a neutral stimulus (NS) with an unconditioned stimulus (US). In general, acquisition proceeds rapidly during early conditioning trials and then gradually levels off. The maximum amount of conditioning that can take place in a particular situation is known as the asymptote of conditioning (refer to Figure 4.1). The asymptote of conditioning, as well as the speed of conditioning, is dependent on several factors. In general, more intense USs produce stronger and more rapid conditioning than do less intense USs. For example, we can obtain stronger conditioning of a salivary response when the US consists of a large amount of food or a highly preferred food than if it consists of a small amount or less preferred food. Likewise, a severe bite from a dog will result Strength of conditioned response (CR) Figure 4.1 A typical acquisition curve in which strength of conditioning increases rapidly during the first few trials and then gradually levels off over subsequent trials. Asymptote of conditioning 1 2 3 4 5 6 7 8 9 10 Number of conditioning trials 11 12 13 14 15 133 134 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Quick Quiz A in a stronger fear than a minor bite will. Similarly, more intense NSs result in stronger and more rapid conditioning than do less intense NSs. For example, a loud metronome that has been paired with food produces a stronger response of salivation than a faint metronome that has been paired with food. And, not surprisingly, conditioned fear responses to dogs are more readily acquired if the person is bitten by a large dog rather than by a small dog. 1. The process of strengthening a conditioned response through repeated pairings of . In general, conditioning an NS with a US is known as proceeds more (rapidly/slowly) during the early trials of a conditioning procedure. 2. The maximum amount of learning (or conditioning) that can take place in a given of learning. situation is known as the 3. In general, a (more/less) intense US produces stronger conditioning, and a (more /less) intense NS produces stronger conditioning. Extinction and Related Phenomena Given that a certain stimulus now elicits a conditioned response, is there any way to eliminate the response? In the process known as extinction, a conditioned response is weakened or eliminated when the conditioned stimulus (CS) is repeatedly presented in the absence of the US. The term extinction also applies to the procedure whereby this happens, namely the repeated presentation of the CS in the absence of the US. Suppose, for example, that a metronome has been paired with food such that it now elicits a conditioned response of salivation: Metronome (NS): Food (US)! Salivation (UR) Metronome (CS)! Salivation (CR) If we now continue to present the metronome by itself and never again pair it with food (each presentation of the metronome being known as an “extinction trial”), the conditioned response of salivation will eventually die out—that is, the CR of salivation will have been extinguished. Metronome (“NS”) ! No Salivation (—) The process of extinction is the decrease in the strength of the CR, and the procedure of extinction is the means by which this is carried out, namely the repeated presentation of the metronome without the food. In a similar manner, if a dog that once bit me never bites me again, my fear response to the dog should eventually disappear. Unfortunately, some people who have been bitten by a dog continue to fear that dog as well as other dogs, in which case we might say that they have a “phobia” of dogs. But if the person has never again been bitten by a dog, why is their fear so persistent? One reason is that people who fear dogs tend to avoid them, and to the extent that they avoid them, their fear response cannot be extinguished. As you will notice in later chapters, this tendency to avoid a feared event is a 4.1 Some Basic Conditioning Phenomena major factor in the development and maintenance of a phobia, and treatment procedures for phobias are often based on preventing this avoidance response from occurring. Once a CR has been extinguished, one should not assume that the effects of conditioning have been completely eliminated. For this reason, in the preceding diagram, the “NS” following extinction has been placed in quotation marks, since it is no longer a pure neutral stimulus. For one thing, a response that has been extinguished can be reacquired quite rapidly when the CS (or NS) is once again paired with the US. If we again pair the metronome with food following an extinction procedure, it may take only a few pairings before we achieve a fairly strong level of conditioning. Likewise, if I somehow manage to overcome my phobia of dogs, I might rapidly reacquire that phobia if I have another frightening experience with dogs. As further evidence that extinction does not completely eliminate the effects of conditioning, an extinguished response can reappear even in the absence of further pairings between the CS and US. Suppose, for example, that we do extinguish a dog’s conditioned salivary response to a metronome by repeatedly presenting the metronome without food. By the end of the extinction session, the metronome no longer elicits salivation. However, if we come back the next morning and sound the metronome, the dog will very likely salivate. In everyday terms, it is almost as if the dog has forgotten that the metronome no longer predicts food. As a result, we are forced to conduct another series of extinction trials, repeatedly sounding the metronome without the food. After several trials, the response is again extinguished. The next day, however, the dog again starts salivating when we first present the metronome. At this point, we might be tempted to conclude that we have an awfully dumb dog on our hands. The dog, however, is simply displaying a phenomenon known as spontaneous recovery. Spontaneous recovery is the reappearance of a conditioned response to a CS following a rest period after extinction. Fortunately, spontaneous recovery does not last forever. Each time the response recovers, it is usually weaker and extinguishes more quickly than before (refer to Figure 4.2). Therefore, after several extinction sessions, we should be able to sound the metronome at the start of the session and find little or no salivation. The phenomenon of spontaneous recovery is particularly important to remember when attempting to extinguish a conditioned fear response. For example, we might arrange for a dog-phobic child to spend several hours with a dog. At the end of that time, the child’s fear of the dog might seem to have been totally eliminated. Nevertheless, it is quite possible that the fear will at least partially recover the next time the child is confronted with a dog, and that several sessions of extinction may be needed before the fear is completely eliminated. Similarly, if you feel terribly anxious with a new date at the start of the evening but more at ease after a couple of hours, do not be surprised if you again find yourself becoming anxious at the start of your next date. It!may take several dates with that person before you feel comfortable right from the outset. Likewise, following a breakup, 135 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Figure 4.2 Hypothetical results illustrating a decline in spontaneous recovery across repeated sessions of extinction. Session 1 Session 2 Session 3 Session 4 Session 5 Session 6 Strength of CR 136 Extinction trials (presentation of CS without US) it may take a while before your feelings of attraction to the other person are finally extinguished, and even then they may intermittently reappear for a considerable period of time. To Pavlov (1927), the phenomenon of spontaneous recovery indicated that extinction is not simply a process of unlearning the conditioning that has taken place. Rather, extinction involves learning something new, namely, to inhibit the occurrence of the CR in the presence of the CS. For example, rather than unlearning the response of salivation to the metronome during extinction, the dog learns to inhibit the response of salivation to the metronome, with the connection between the metronome and salivation still remaining intact on some underlying level. Spontaneous recovery may therefore represent the partial weakening of this inhibition during the rest period between extinction sessions. Support for the notion that extinction involves a buildup of inhibition is also provided by a phenomenon known as disinhibition. Disinhibition is the sudden recovery of a conditioned response during an extinction procedure when a novel (unfamiliar) stimulus is introduced. For example, if we are in the process of extinguishing conditioning to a metronome but then present an unusual humming noise in the background, the sound of the metronome may again elicit a considerable amount of salivation. In diagram form: (Step 1: First condition the metronome as a CS for salivation) Metronome (NS): Food (US) ! Salivation (UR) Metronome (CS) ! Salivation (CR) (Step 2: Begin extinction trials by repeatedly presenting the metronome by itself; as a result, the CR is greatly weakened) Metronome (CS) ! Weak salivation (partially extinguished CR) 4.1 Some Basic Conditioning Phenomena 137 (Step 3: Present a novel humming noise in background during the extinction procedure; as a result, the CR recovers in strength) (Unusual humming noise) Metronome (CS) ! Salivation (full CR) Similarly, if your anxiety while giving a speech in class gradually fades, it may suddenly recover when a noisy ceiling fan starts up or someone walks in late. (Note that the phenomenon of disinhibition is similar to dishabituation, discussed in Chapter 3, in which the presentation of a novel stimulus results in the reappearance of a habituated response. To distinguish these concepts, it may help to remember that dishabituation involves the reappearance of a habituated response, and disinhibition involves the recovery of a response that has become partially inhibited due to extinction.)1 Note that extinction is not the same as forgetting. Consider, for example, a young child who is scratched by a cat and as a result becomes slightly fearful of it. If they do not see the cat again for several years, cannot consciously recall the incident, and show no fear of the cat when they do see it, it is quite likely that they simply forgot about the incident. Extinction, however, is where the child does remain afraid of the cat several years later (whether or not they consciously recall the incident) and requires repeated exposure to the cat (the CS) without being scratched (the US) in order for their fear of the cat to be eliminated. The key difference, therefore, is that in extinction, a CR is eliminated because of repeated exposure to the CS in the absence of the US, whereas forgetting simply requires the passage of time and does not require repeated exposure to the CS in order for the CR to be eliminated. ________________________________________________________________. 2. The procedure of extinction involves ____________________________________ _________________________________________________________________. 3. Once a CR has been extinguished, reacquisition of that response tends to occur (more/less) rapidly than the original conditioning. 4. The sudden recovery of an extinguished response following some delay after extinction is known as s____________________ r____________________. 5. With repeated sessions of extinction, each time a response recovers, it is usually somewhat (weaker/stronger) and extinguishes more (slowly/quickly). 6. Pavlov believed that this phenomenon indicates that extinction involves the (inhibition/unlearning) of a conditioned response. 7. The sudden recovery of a response during an extinction procedure when a novel stimulus is introduced is called ____________________. 1 Another reason that extinction of CRs can be difficult or incomplete is that we might not identify all of the stimuli that are helping to elicit the response. For example, in the case of a dog phobia, stimuli other than the dog—such as the sound of growling or worried looks on the faces of others—may also be involved. Those stimuli may also need to be incorporated into the extinction procedure. Quick Quiz B 1. In the process of extinction, a conditioned response grows weaker because 138 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Stimulus Generalization and Discrimination Classical conditioning would not be very useful if it only enabled us to learn about relationships between particular stimuli. For example, if we were bitten by a spider, it would not be very helpful for us to fear only that particular spider (which, in any case, we probably obliterated the moment it bit us). From an evolutionary perspective, it would be far more adaptive to learn to fear other spiders as well, particularly spiders that look similar to the one that bit us. Fortunately, this is precisely what happens, through a process known as stimulus generalization. In classical conditioning, stimulus generalization is the tendency for a CR to occur in the presence of a stimulus that is similar to the CS. In general, the more similar the stimulus is to the original CS, the stronger the response. For example, if a dog is conditioned to salivate to a tone that has a pitch of 2000!Hz, it will salivate to similar tones as well. But it will salivate more strongly to a 1900-Hz tone or a 2100-Hz tone than it will to a 1000-Hz tone or a 3000-Hz tone. In other words, tones that are most similar in pitch to the original CS will elicit the strongest response. Similarly, after being bitten by a dog, a child will probably fear not only that particular dog but other dogs as well. And the child is particularly likely to fear dogs that closely resemble the dog that bit him. The process of generalization is most apparent when the stimuli involved are physically similar and vary along a continuum. Tones of varying pitch or loudness and lights of varying color or brightness are examples of such stimuli. However, generalization can also occur across nonphysical dimensions, particularly in humans who use language. Semantic generalization is the generalization of a conditioned response to verbal stimuli that are similar in meaning to the CS. For example, if humans are exposed to a conditioning procedure in which the sight of the word car is paired with shock, that word eventually becomes a CS that elicits a fear response. When participants are shown other words, generalization of the fear response is more likely to occur to those words that are similar in meaning to car, such as automobile or truck, than to words that look or sound similar, such as bar or tar. Thus, the meaning of the word is the critical factor in semantic generalization. For this reason, words that have similar meaning for an individual—for example, Jennifer Lopez and J. Lo—are likely to generate the same conditioned emotional response. The opposite of stimulus generalization is stimulus discrimination, the tendency for a response to be elicited more by one stimulus than another. For example, if the dog salivates in the presence of the 2000-Hz tone but not in the presence of a 1900-Hz tone, then we say that it is able to discriminate, or has formed a discrimination, between the two stimuli. Such discriminations can be deliberately trained through a procedure known as discrimination training. If we repeatedly present the dog with one type of trial in which a 2000-Hz tone is always followed by food and another type of trial in which a 1900-Hz tone is never followed by food, the dog will soon learn to 4.1 Some Basic Conditioning Phenomena salivate in the presence of the 2000-Hz tone and not in the presence of the 1900-Hz tone. Step 1: Conditioning phase (with the two types of trials presented several times in random order) 2000-Hz tone (NS): Food (US) ! Salivation (UR) 1900-Hz tone (NS): No Food (—) Step 2: Test phase 2000-Hz tone (CS1) ! Salivation (CR) 1900-Hz tone (CS2) ! No salivation (—) As a result of training, the 2000-Hz tone has become an excitatory CS (or CS1) because it predicts the presentation of food, and the 1900-Hz tone has become an inhibitory CS (or CS–) because it predicts the absence of food. The discrimination training has, in effect, countered the tendency for generalization to occur. (Note that the two types of trials were presented in random order during the conditioning phase. If they were instead presented in alternating order, the dog might associate the presentation of food with every second tone rather than with the tone that has a pitch of 2000 Hz.) As you may have already guessed, discrimination training is a useful means for determining the sensory capacities of animals. For example, by presenting an animal with a CS1 tone and a CS– tone that are successively more and more similar, we can determine the animal’s ability to discriminate between tones of different pitch. If it salivates to a CS1 of 2000 Hz and does not salivate to a CS– of 1950 Hz, then it has shown us that it can distinguish between the two. But if it salivates equally to both the 2000-Hz tone and the 1950-Hz tone, then it cannot distinguish between the two. Generalization and discrimination play an important role in many aspects of human behavior. Phobias, for example, involve not only the classical conditioning of a fear response, but also an overgeneralization of that fear response to inappropriate stimuli. For example, a woman who has experienced an abusive relationship with a man may develop feelings of anxiety and apprehensiveness toward most men. Eventually, however, through repeated interactions with men who are not abusive, this tendency can decrease and she can begin to adaptively discriminate between men who are potentially abusive and those who are not. Unfortunately, such discriminations are not always easily made, and further bad experiences could greatly strengthen her fear. Moreover, if the woman begins to avoid all men, then the tendency to overgeneralize may remain, thereby significantly impairing her social life. As noted earlier, if we avoid that which we are afraid of, it is difficult for us to overcome our fears. 139 Quick Quiz C 140 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities 1. Stimulus generalization is the tendency for a (CR/UR) to occur in the presence 2. 3. 4. 5. of stimuli that are similar to the original (CS/US). In general, the more (similar /different) the stimulus, the stronger the response. The generalization of a conditioned response to stimuli that are similar in generalization. meaning to a verbal CS is called s The opposite of stimulus generalization is stimulus . This can be defined as . Feeling anxious around all objects that look like a rattlesnake is an example of , whereas feeling anxious around rattlesnakes stimulus . but not other snakes is an example of stimulus Suppose Cary disliked his physics instructor and, as a result, came to dislike all science instructors. This example illustrates the process of . over Experimental Neurosis Overgeneralization is not the only way that processes of discrimination versus generalization influence the development of a psychological disorder. For example, Pavlov (1927, 1928) reported an interesting discovery made by a colleague, Shenger-Krestovnikova, that arose during a discrimination training procedure. In this experiment, an image of a circle signaled the presentation of food and an ellipse signaled no food (refer to Figure 4.3). In keeping with normal processes of discrimination, the dog duly learned to salivate when it saw the circle (a CS1) and not salivate when it saw the ellipse (a CS–). Following this, the ellipse was gradually made more circular, making it more difficult for the dog to determine when food was about to appear. When the ellipse was almost completely circular, the dog was able to make only a weak discrimination, salivating slightly more in the presence of the circle than in the presence of the ellipse. Interestingly, continued training with these stimuli did not result in any improvement. In fact, after several weeks, the discrimination was lost. More interestingly, however, the hitherto well-behaved dog became extremely agitated during each session—squealing, wriggling about, and biting at the equipment. It acted as though it were experiencing a serious emotional breakdown. Pavlov called this phenomenon experimental neurosis, an experimentally produced disorder in which animals exposed to unpredictable events develop neurotic-like symptoms. Pavlov hypothesized that human neuroses might Figure 4.3 Discrimination training procedure used by Shenger-Krestovnikova in which the picture of a circle functioned as the CS1 and the picture of the ellipse functioned as the CS–. : Food : No food Salivation 4.1 Some Basic Conditioning Phenomena develop in a similar manner. Situations of extreme uncertainty can be stressful, and prolonged exposure to such uncertainty might result in the development of neurotic symptoms. Thus, in the opening vignette to this chapter, it is not surprising that Jana’s romantic partners often display increasing symptoms of neuroticism as the relationship progresses. A little uncertainty in one’s romantic relationships can be exciting, but extreme uncertainty might eventually become aversive. In carrying out their studies of experimental neurosis, Pavlov and his assistants also discovered that different dogs displayed different symptoms. Some dogs became anxious when exposed to the procedure, while others became catatonic (rigid) and acted almost hypnotized. A few dogs, however, displayed few if any symptoms and did not have a breakdown. Pavlov speculated that such differences reflected underlying differences in temperament. This was an extension of one of Pavlov’s earlier observations that certain dogs condition more easily than others. Shy, withdrawn dogs seemed to make the best subjects, conditioning easily, whereas active, outgoing dogs were more difficult to condition (which is quite the opposite of what Pavlov had originally expected). Based on results such as these, Pavlov formulated a theory of personality in which inherited differences in temperament interact with classical conditioning to produce general patterns of behavior. Eysenck (1957) later utilized certain aspects of Pavlov’s work in formulating his own theory of personality. A major aspect of Eysenck’s theory is the distinction between introversion and extroversion. In very general terms, introverts are individuals who are highly reactive to external stimulation (hence, cannot tolerate large amounts of stimulation and tend to withdraw from such stimulation), condition easily, and develop anxiety-type symptoms in reaction to stress. By contrast, extroverts are less reactive to external stimulation (hence, can tolerate, and will even seek out, large amounts of stimulation), condition less easily, and develop physical-type symptoms in reaction to stress. Eysenck’s theory also proposes that psychopaths, individuals who engage in antisocial behavior, are extreme extroverts who condition very poorly. As a result, they experience little or no conditioned anxiety when harming or taking advantage of others, such anxiety being the underlying basis of a conscience. Both Pavlov’s and Eysenck’s theories of personality are considerably more complicated than presented here, involving additional dimensions of personality and finer distinctions between different types of conditioning, especially excitatory and inhibitory conditioning. Thus, extroverts do not always condition more poorly than introverts, and additional factors are presumed to influence the development of neurotic symptoms (Clark, Watson, & Mineka, 1994; Eysenck, 1967; Monte, 1999). Nevertheless, processes of classical conditioning interacting with inherited differences in temperament could well be major factors in determining one’s personality. The experimental neurosis paradigm also suggests that prolonged exposure to unpredictable events can sometimes have serious effects on our well-being. We will explore this topic in more detail in Chapter 9. 141 Quick Quiz D 142 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities 1. In Shenger-Krestovnikova’s experiment, the animal displayed symptoms 2. 3. 4. 5. 6. 7. consistent with an emotional breakdown when exposed to a CS1 and a CS2 that were made progressively (more/less) similar. Pavlov referred to this breakdown as e n , an experimentally produced disorder in which animals exposed to unp____________________ events develop n_________________-like symptoms. Pavlov and his assistants noted that the dogs displayed two general patterns , while other dogs of symptoms. Some dogs became . In addition, (all/not all) dogs developed became symptoms. Pavlov believed that these differences between dogs in the types of symptoms they . developed reflected (learned/inherited) differences in t In Eysenck’s theory, introverts are (more/less) reactive to external stimulation than extroverts are, and they therefore (can/cannot) tolerate large doses of stimulation. Introverts also condition (more/less) easily than extroverts. Introverts seem to develop a -type symptoms in reaction -type symptoms. to stress, whereas extroverts develop p According to Eysenck, psychopaths tend to be extreme (introverts/extroverts) who condition (very easily/very poorly). They therefore feel little or no conditioned when harming or manipulating others. Study Tip: The extent to which introverts versus extroverts react to external stimulation also has implications for the type of environment in which one can effectively study. In general, extroverts, being less reactive to external stimulation, are better able to cope with potential distractors, like background noise or loud music, and may even prefer such environments. Introverts, on the other hand, being more reactive to external stimulation, function better in environments that are relatively quiet or that have soft music playing in the background. The most common finding, however, is that both introverts and extroverts perform best in environments that have minimal distractions, with no music (e.g., Chamorro-Premuzic, Swami, Terrado, & Furnham, 2009). That said, remember that such studies are reporting average tendencies found among groups of people; thus, these findings may or may not apply to you as an individual. As with self-management in general, the key is to discover what works best for you. For example, I (Russ Powell) once ran into a top student who didn’t seem particularly extroverted but was nevertheless studying in the food fair area of a busy mall! She said it was her favorite place to study; it seemed as though the constant background noise maintained her arousal at a high enough level that studying there never became monotonous for her. Conversely, my coauthor, Lynne, a rather extreme extrovert, has mentioned that she learns well with almost any kind of background music, loud or soft, but heaven forbid that it include singing; words are extremely distracting to her. I too prefer instrumental music while I am working, but it has to be very soft music, and even then I sometimes have to turn it off. So what kind of environment is best for you to study in? 4.2 Three Extensions to Classical Conditioning 4.2 Three Extensions to Classical Conditioning The normal classical conditioning procedure involves associating a single neutral stimulus with a US. But stimuli rarely exist in isolation. For example, a friend does not exist as an isolated element in a child’s world. The friend is associated with a variety of other stimuli, such as the house they live in, the books they like to read, and the other kids they hang around with. If a child makes a new friend and learns to have positive associations with them, will they also develop positive feelings toward the various objects, places, and people with which their friend is associated? In more technical terms, can classical conditioning of a CS also result in the development (or strengthening) of a conditioned response in the absence of a direct pairing between the NS (or CS) and the US? The processes outlined in this section indicate that it can. Higher-Order Conditioning Suppose you are stung by a wasp while out for a run one day and, as a result, develop a terrible fear of wasps. Imagine, too, that following the development of this fear, you notice a lot of wasps near the trash bin outside your apartment building. Could the trash bin also come to elicit a certain amount of fear, or at least a feeling of edginess or discomfort? In higher-order conditioning, a stimulus that is associated with a CS can also become a CS. Thus, the trash bin could very well come to elicit a fear response through its association with wasps. This process can be diagrammed as follows: (Step 1: Basic conditioning of a fear response to wasps. As part of a higher-order conditioning procedure, this first step is called “first-order conditioning,” and the original NS and CS are respectively labeled NS1 and CS1.) Wasp (NS1): Sting (US)! Fear (UR) Wasp (CS1) ! Fear (CR) (Step 2: Higher-order conditioning of the trash bin through its association with wasps. This second step is sometimes also called “secondorder conditioning,” and the new NS and CS are labeled NS2 and CS2.) Trash bin (NS2): Wasp (CS1)! Fear (CR) Trash bin (CS2): ! Fear (CR) The CS2 generally elicits a weaker response than the CS1 (which, as noted in Chapter 3, generally elicits a weaker response than the US). Thus, the fear response produced by the trash bin is likely to be much weaker than the fear response produced by the wasps. This is not surprising, given that the trash 143 144 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Figure 4.4 In this example of higher-order conditioning, a metronome is paired with food and becomes a CS1 for salivation, following which a light paired with the metronome becomes a CS2 for salivation. (Source: Nairne, 2000.) Metronome Food (followed by) NS1 Salivation (elicits) US Metronome UR Salivation (elicits) CR CS1 Light Metronome (followed by) CS1 NS2 Light (elicits) CS2 Salivation (elicits) CR Salivation CR bin is only indirectly associated with the unconditioned stimulus (i.e., the wasp sting) upon which the fear response is based. An experimental example of higher-order conditioning might involve pairing a metronome with food so that the metronome becomes a CS1 for salivation, and then pairing a light with the metronome so that the light becomes a CS2 for salivation (refer to Figure 4.4). In diagram form: (Step 1: First-order conditioning) Metronome (NS1): Food (US)! Salivation (UR) Metronome (CS1): ! Salivation (CR) 4.2 Three Extensions to Classical Conditioning 145 (Step 2: Second-order, or higher-order, conditioning) Light (NS2): Metronome (CS1)! Salivation (CR) Light (CS2): ! Salivation (CR) The light now elicits salivation although it has never been directly paired with food. (For consistency, we will continue to use Pavlov’s salivary conditioning procedure as the basic experimental example throughout much of this chapter. In reality, however, modern researchers use other procedures to study classical conditioning, such as the conditioned suppression procedure discussed in Chapter 3.) We could also attempt third-order conditioning by pairing yet another stimulus, such as the sound of a tone, with the light. However, third-order conditioning can be experimentally difficult to obtain, and when it is obtained, the conditioned response to a third-order conditioned stimulus (the CS3) is likely to be quite weak. Higher-order conditioning is commonly used in advertising. For example, advertisements often pair a product with celebrities who, through past conditioning, have come to elicit positive emotional responses in many of us. As discussed in the And Furthermore box, the assumption is that the sight of the celebrity will elicit a positive emotional response in us that will be associated with the product and thereby increase the likelihood that we will purchase the product. conditioning, an already established CS is used to condition a new CS. 2. In general, in this type of conditioning procedure, the CS2 elicits a (weaker/ stronger) response than the CS1. 3. In higher-order conditioning, conditioning of the CS 1 is often called -order conditioning, while conditioning of the CS2 is -order conditioning. called 4. In a higher-order conditioning procedure in which a car is associated with a famous person, the famous person is the (CS1/CS2) and the car is the (CS1/CS2). Sensory Preconditioning We have observed that an event that is subsequently associated with a feared event like wasps, such as trash bins, can become a CS for fear. What about an event that was previously associated with wasps, such as a toolshed that once had a wasps’ nest hanging in it? Will walking near the shed now also elicit feelings of anxiety? In sensory preconditioning, when one stimulus is conditioned as a CS, another stimulus with which it was previously paired can also become a CS. If you previously associated the toolshed with wasps and then acquired a Quick Quiz E 1. In 146 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities And Furthermore When Celebrities Misbehave Robert Alexander/Archive Photos/Getty Images As mentioned, advertisers are aware that we are more likely to buy products that are associated with celebrities . . . but will any celebrity do? Some companies will shy away from a celebrity who has been convicted of a crime or implicated in some sort of scandal. For example, when the U.S. AntiDoping Agency revealed in 2012 that Lance Armstrong had used banned performance-enhancing drugs, the seven-time Tour de France winner was stripped of his awards, was banned from competitive cycling, and lost an estimated $150 million in endorsement deals. His sponsors, including 24-Hour Fitness and Oakley, were quick to denounce his use of drugs in competition (Rotunno, 2012). When actor Ko Chen-tung (also known as Kai Ko) was arrested in Beijing for recreational drug use in 2014, he quickly lost endorsement deals with Canon, KFC, Nivea, and Quaker Oats. Companies like Nivea or Quaker Oats are particularly sensitive to indiscretions by their celebrity endorsers because their corporate image is aimed at being “wholesome” or “family oriented.” Do all advertisers react this way to celebrity scandal? Not necessarily. When photos were published in 2005 by the Daily Mirror (a British tabloid) that showed model Kate Moss allegedly using cocaine, she immediately lost some lucrative endorsements with fashion companies, including H&M and Burberry, as well as several modeling contracts. Interestingly, this short-term loss was not sustained; according to Forbes.com (a leading business and finance news site), not only did Burberry re-sign Moss to an endorsement deal, but other high-end clients were quick to sign her to new contracts. Why would companies want their products associated with a drug-using model? The fashion industry thrives on what is “edgy,” and many designers and retailers want their products to be associated with things that are dark and dangerous, as well as sexy. While some companies (like H&M, which made statements about its antidrug stance in the wake of the Moss cocaine scandal) try to maintain a clean image, others are comfortable being associated with the darker side of life. Thus, if consumers associate a product with 4.2 Three Extensions to Classical Conditioning 147 the dangerous and less-than-pure image of Kate Moss, then they are making exactly the association the retailer was hoping for. (And they can put on that eyeliner, and feel a little bit dangerous, without having to resort to cocaine use of their own!) In 2016, as a peaceful protest against racism in policing, professional football quarterback Colin Kaepernick kneeled while the national anthem was playing before his games. Controversy erupted among football fans and the broader society, and it seemed that no NFL team was willing to sign Kaepernick when he opted out of his contract with the San Francisco 49ers. Without a contract, an athlete usually has no hope of a corporate endorsement, but Nike signed Kaepernick in 2018 to be the face of a new campaign. Nike’s sales surged, and their stock rose 5% in the weeks following the campaign launch despite outrage and protests by those opposed to Kaepernick’s public stance (Nike’s “Dream Crazy,” 2019). fear of wasps as a result of being stung, you might also feel anxious when walking near the toolshed. This process can be diagrammed as follows: (Step 1: Preconditioning phase in which the toolshed is associated with wasps) Toolshed (NS2): Wasps (NS1) (Step 2: Conditioning of wasps as a CS1) Wasp (NS1): Sting (US) ! Fear (UR) Wasp (CS1) ! Fear (CR) (Step 3: Presentation of the toolshed) Toolshed (CS2) ! Fear (CR) The toolshed now elicits a fear response, although it was never directly paired with a wasp sting. An experimental example of sensory preconditioning with dogs involves first presenting several pairings of two neutral stimuli, such as a light and a metronome. The metronome is then paired with food to become a CS for salivation. As a result of this conditioning, the light, which has never been directly paired with the food but has been associated with the metronome, also comes to elicit salivation (refer to Figure 4.5). This process can be diagrammed as follows: 148 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Figure 4.5 In this example of sensory preconditioning, a dog is presented with several pairings of a light and a metronome. The metronome is then paired with food and becomes a conditioned stimulus for salivation. As a result, the light that was previously paired with the metronome also becomes a conditioned stimulus for salivation. (Source: Nairne, 2000.) Metronome No Salivation Light (followed by) NS2 NS1 Metronome (followed by) Food NS1 (elicits) US Salivation UR Metronome (elicits) CS1 Light CS2 Salivation CR (elicits) Salivation CR (Step 1: Preconditioning phase, in which the light is repeatedly associated with the metronome) Light (NS2): Metronome (10 presentations of this pairing) (NS1) (Step 2: Conditioning of the metronome as a CS1) Metronome (NS1): Food (US) ! Salivation (UR) Metronome (CS1) ! Salivation (CR) 4.2 Three Extensions to Classical Conditioning (Step 3: Presentation of the light) Light (CS2) ! Salivation (CR) As with higher-order conditioning, the response elicited by the light (CS2) is generally weaker than the response elicited by the metronome (CS1). Likewise, the fear response elicited by the toolshed (CS 2) is likely to be weaker than the fear response elicited by the wasps (CS1). Although it was once believed necessary to pair the neutral stimuli hundreds of times in the preconditioning phase (e.g., Brogden, 1939), it is now known that this type of conditioning works best if the stimuli are paired relatively few times (R.!F. Thompson, 1972). This prevents the animal from becoming overly familiar with the stimuli prior to conditioning (you will later learn that this can interfere with conditioning). Another unusual finding with sensory preconditioning is that the procedure is sometimes more effective when the two stimuli in the preconditioning phase are presented simultaneously as opposed to sequentially (Rescorla, 1980). This result is unusual because it contradicts what we find with NS–US pairings, in which simultaneous presentation of the two stimuli is relatively ineffective. Sensory preconditioning is significant because it demonstrates that stimuli can become associated with each other in the absence of any identifiable response (other than an orienting response). In this sense, sensory preconditioning can be viewed as a form of latent learning, which was first discussed in Chapter 1. Just as Tolman’s rats learned to find their way around a maze even when it seemed as if there were no significant consequences for doing so (i.e., food had not yet been introduced into the goal box), animals will associate stimuli with each other even when those stimuli seem to have little significance for them. US Revaluation At the beginning of this chapter, we mentioned how more intense stimuli produce stronger conditioning than do less intense stimuli. For example, a strong shock will produce stronger fear conditioning than a weak shock does. But what would happen if we conducted our conditioning trials with one level of shock and then presented a different level of shock by itself on a subsequent nonconditioning trial? In other words, would changing the intensity or value of the US after the conditioning of a CS also change the strength of response to the CS? Imagine, for example, that the sound of a metronome is followed by a small amount of food, with the result that the metronome comes to elicit a small amount of saliva. Metronome (NS): Small amount of food (US) ! Weak Salivation (UR) Metronome (CS) ! Weak Salivation (CR) 149 150 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Once this conditioning has been established, we now present the dog with a large amount of food, which elicits a large amount of saliva. Large amount of food (US) ! Strong Salivation (UR) What type of response will now be elicited by the metronome? As it turns out, the dog may react to the metronome as though it predicts a large amount of food rather than a small amount of food. Metronome (CS) ! Strong Salivation (CR) Quick Quiz F Note that the metronome was never directly paired with the large amount of food; the intervening experience with the large amount by itself produced the stronger CR of salivation. Therefore, US revaluation is the postconditioning presentation of the US at a different level of intensity, thereby subsequently altering the strength of response to the previously conditioned CS. It is called US revaluation because the value or magnitude of the US is being changed. Depending on whether the US is increased or decreased in strength, this procedure can also be called US inflation or US deflation. The preceding scenario is an example of US inflation. As an example of US deflation, imagine that you salivate profusely when you enter Joe’s Restaurant because you love their turkey gumbo. You then get a new roommate, who, as it turns out, is a turkey gumbo fanatic and prepares turkey gumbo five times a week. Needless to say, so much turkey gumbo can become monotonous (an instance of longterm habituation), and you finally reach a point where you have little interest in turkey gumbo. As a result, when you next enter Joe’s Restaurant, you salivate very little. The value of turkey gumbo has been markedly reduced in your eyes, which in turn affects your response to the restaurant that has been associated with it. In everyday terms, it seems like what is happening in US revaluation is that the animal has learned to expect the US whenever it sees the CS; the intensity of its response is thus dependent on the animal’s most recent experience with the US. This interpretation fits well with cognitive explanations of conditioning, as will be discussed in the next chapter. (You will also note that US revaluation can play a role in the development or exacerbation of phobias.) 1. Suppose you suddenly developed a strong fear of dogs after being severely bitten. As a result, you are also anxious about cycling because, on several occasions in the past, you witnessed people walking their dogs on the cycling paths. This example illustrates the phenomenon of . 2. In the earlier example, the cycling paths will probably elicit a (stronger/weaker) fear response than will the sight of the dogs. 3. Sensory preconditioning often works best when the two neutral stimuli are paired (relatively few/hundreds of) times in the preconditioning phase. 4.3 Four Examples of Specificity in Classical Conditioning 4. Unlike NS–US pairings in normal conditioning, NS–NS pairings can result in stronger sensory preconditioning when the two stimuli are presented (sequentially/simultaneously). 5. The (US/CS) r procedure involves the (preconditioning/ postconditioning) presentation of the (CS/US) by itself at a different level of intensity than used in the original conditioning. 6. Nikki feels excited when she sees her father arrive home each evening because he always brings her licorice. One day, her mother bought her licorice earlier in the day, and Nikki had no desire for licorice when evening came around. As a result, she was not as excited when her father came home that evening. In this example, her father is a (CS/US) through his association with licorice. Being satiated with licorice therefore reduced the value of the (CS/US) that typically followed her father’s arrival. As a result, her (CR/UR) of excitement on seeing her father was greatly reduced. 4.3 Four Examples of Specificity in!Classical Conditioning In the preceding section, we examined three ways in which the classical conditioning can be extended or strengthened even in the absence of a direct pairing between an NS (or CS) and a US. In this section, we discuss four procedures—overshadowing, blocking, occasion setting, and latent inhibition—that demonstrate that conditioning doesn’t automatically occur simply because an NS and a US have been paired with each other; rather, conditioning often occurs to specific stimuli or in specific circumstances only. Two of these procedures (overshadowing and blocking) involve the presentation of what is known as a compound stimulus. A compound stimulus consists of the simultaneous presentation of two or more individual stimuli (e.g., the sound of a metronome is presented at the same time as a light). Overshadowing If you were stung by a wasp during a walk in the woods, would it make sense to develop a conditioned fear response to every stimulus associated with that event (e.g., the trees surrounding you, the butterfly fluttering by, and the cloud formation in the sky)? No, it would not. Rather, it would make more sense to develop a fear of those stimuli that were most salient (that really stood out) at the time of being stung, such as the sight of the wasp. In overshadowing, the more salient member of a compound stimulus is more readily conditioned as a CS and thereby interferes with conditioning of the less salient member. In the wasp example, you are likely to develop a conditioned fear response to the most distinctive stimuli associated with that event, such as the sight of the wasp and perhaps the buzzing sound it makes. An experimental example of overshadowing might involve first pairing a compound stimulus, such as a bright light and a faint-sounding metronome, with food. After several pairings, the compound stimulus becomes a CS that 151 152 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Figure 4.6 In this example of overshadowing, a bright light and a faintsounding metronome are simultaneously presented as a compound stimulus and paired with food. After several pairings, the compound stimulus becomes a CS that elicits salivation. However, when each member of the compound is tested separately, the bright light elicits salivation but the faint-sounding metronome does not. (Source: Nairne, 2000.) Simultaneous presentation Bright light Faint metronome Salivation Food + NS US UR Simultaneous presentation Bright light Faint metronome Salivation + CR CS Bright light CS Faint metronome Salivation CR No salivation NS elicits salivation. However, when each member of the compound is tested separately, the bright light elicits salivation, while the faint metronome elicits little or no salivation (refer to Figure 4.6). In diagram form: 4.3 Four Examples of Specificity in Classical Conditioning (Step 1: Conditioning of a compound stimulus as a CS. Note that the compound stimulus consists of the simultaneous presentation of the twobracketed stimuli.) [Bright light 1 Faint metronome] (NS): Food (US) ! Salivation (UR) [Bright light 1 Faint metronome] (CS) ! Salivation (CR) (Step 2: Presentation of each member of the compound separately) Bright light (CS) ! Salivation (CR) Faint metronome (NS) ! No salivation (—) Due to the presence of the bright light during the conditioning trials, no conditioning occurred to the faint metronome. This is not because the faint metronome is unnoticeable. If it had been paired with the food by itself, it could easily have become an effective CS. Only in the presence of a more salient stimulus does the less salient stimulus come to elicit little or no response. Managers make use of the overshadowing effect when they assign an assistant to announce an unpopular decision. Although the employees might recognize that the manager is mostly responsible, the assistant is the most salient stimulus and will, as a result, bear the brunt of the blame. It is thus the assistant who is likely to become most disliked by the employees. On the other hand, managers often make a point of personally announcing popular decisions, thereby attracting most of the positive associations to themselves even if they have been only minimally involved in those decisions. Similarly, the positive feelings generated by the music of a rock band will be most strongly associated with the most salient member of that band (usually the lead singer)—a fact that often leads to problems when other band members conclude that they are not receiving their fair share of the accolades. Blocking The phenomenon of overshadowing demonstrates that, in some circumstances, mere contiguity between a neutral stimulus and a US is insufficient for conditioning to occur. An even clearer demonstration of this fact is provided by a phenomenon known as blocking. In blocking, the presence of an established CS during conditioning interferes with conditioning of a new CS. Blocking is similar to overshadowing, except that the compound consists of a neutral stimulus and a CS rather than two neutral stimuli that differ in salience. For example, suppose that a light is first conditioned as a CS for salivation. If the light is then combined with a metronome to form a compound, and this compound is then paired with food, little or no conditioning occurs to the metronome (refer to Figure 4.7). 153 154 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Figure 4.7 In this example of blocking, a light is first conditioned as a CS for salivation. When the light is then combined with a metronome to form a compound stimulus, and this compound stimulus is paired with food, the metronome does not become a CS. The presence of the already established CS blocks conditioning to the metronome. (Source: Nairne, 2000.) Light Salivation Food NS US Light UR Salivation CR CS Simultaneous presentation Food Salivation + CS NS US Light Salivation CR CS Metronome NS No salivation UR 4.3 Four Examples of Specificity in Classical Conditioning In diagram form: (Step 1: Conditioning of the light as a CS) Light (NS): Food (US) ! Salivation (UR) Light (CS) ! Salivation (CR) (Step 2: Several pairings of a compound stimulus consisting of the CS and an NS with the US) [Light (CS) 1 Metronome (NS)]: Food (US) ! Salivation (UR) (Step 3: Presentation of each member of the compound separately; the question at this point is whether conditioning occurred to the metronome) Light (CS) ! Salivation (CR) Metronome (“NS”) ! No salivation (—) In step 2, the presence of the light blocked conditioning to the metronome. An everyday (but overly simplistic) way of thinking about what is happening here is that the light already predicts the food, so the dog pays attention only to the light. As a result, the metronome does not become an effective CS despite being paired with the food. For a real-life example of the blocking effect, imagine that you have to make an unpopular announcement to your employees. The phenomenon of blocking suggests that you would do well to make it a joint announcement with another manager who is already disliked by the employees (one who is already an aversive CS). The employees might then attribute most or all of the bad news to the unpopular manager, and you will be left relatively unscathed. The phenomenon of blocking garnered a lot of attention when it was first demonstrated (Kamin, 1969). It clearly indicates that mere contiguity between an NS and a US is insufficient to produce conditioning. Rather, it seems that a more crucial factor in conditioning is the extent to which the NS comes to act as a signal or predictor of the US. In more cognitive terms (Tolman would have loved blocking), the act of conditioning can be said to produce an “expectation” that a particular event is about to occur. When the light is conditioned as a CS, the dog comes to expect that food will follow the light. Later, when the metronome is presented at the same time as the light, the metronome provides no additional information about when food will occur; hence, no conditioning occurs to it. We will again encounter 155 156 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Quick Quiz G this notion of expectations when we discuss the Rescorla-Wagner theory of conditioning in Chapter 5.2 1. A compound stimulus consists of the (simultaneous/successive) presentation of two or more separate stimuli. , the most salient member of a compound stimulus is more readily conditioned as a CS and thereby interferes with conditioning of the , the presence of an established less salient member. In CS within a compound interferes with conditioning of another stimulus. 3. In a(n) procedure, the compound stimulus consists of a procedure, neutral stimulus and a CS, whereas in a(n) the compound stimulus consists of two neutral stimuli that differ in salience or intensity. 4. Because Jez has a history of getting into trouble, he often catches most of the blame when something goes wrong, even when others are also responsible. This is . most similar to the phenomenon of 2. In Occasion Setting As we have learned, classical conditioning involves the establishment of an association between two events, such as between the sound of a metronome and the taste of food or between the sight of a wasp and the feel of its sting. To date, however, we have largely ignored the fact that these two events do not exist in isolation but instead occur within a certain context. This context often comes to serve as an overall predictor of the relationship between these two events. To give a fanciful example, imagine that Dr. Pavlov’s dogs often see him throughout the day, both during and outside of conditioning trials. As such, the appearance of Dr. Pavlov is not always followed by food, and the dogs therefore do not salivate when they see him. Also imagine, however, that during experimental sessions, whenever the dogs hear the sound of a metronome, it is followed by food, but only when Dr. Pavlov is present. When Dr. Pavlov is absent, the sound of the metronome is not followed by food. Hence: Dr. Pavlov present {Metronome (NS): Food (US) ! Salivation (UR) Dr. Pavlov absent {Metronome (NS): No food (—) 2 A different way of thinking about this (again, popular with researchers who have a preference for cognitive interpretations of such matters) is that increases in conditioning can occur only to the extent that a US is unexpected or surprising. Once a US is fully expected, such as when a light by itself reliably predicts the occurrence of food, no further conditioning can occur. In more general terms, we learn the most about something when we are placed in a position of uncertainty and must then strive to reduce that uncertainty. Once the uncertainty has been eliminated, learning ceases to occur. Thus, in blocking, no conditioning (no new learning) occurs to the neutral stimulus, because the presence of the CS that it has been combined with ensures that the animal is not surprised when the US soon follows. 4.3 Four Examples of Specificity in Classical Conditioning The metronome therefore predicts the delivery of food, but only when Dr. Pavlov is present and not when he is absent. As a result, the metronome elicits salivation (it becomes a CS) only when Dr. Pavlov is present and not when he is absent: Dr. Pavlov present (OS) {Metronome (CS): ! Salivation (CR) Dr. Pavlov absent {Metronome (“NS”): ! No salivation (—) (“OS” stands for occasion setter.) Dr. Pavlov in this instance is called an occasion setter because his presence predicts the occasions on which the metronome is followed by food. He himself does not elicit salivation, but his presence controls the extent to which the metronome acts as a CS that elicits salivation. Thus, occasion setting is a procedure in which a stimulus (an occasion setter) signals whether a stimulus is likely to be followed by a US and thereby determines whether that stimulus will act as a CS that elicits a CR. An occasion setter can signal either the presentation of a US or its absence, or even a change in the intensity of the US. Imagine, for example, that a child is frequently scolded or subjected to other mild punishments; however, when their parents are drinking alcohol, punishments are more frequent and severe. Thus: Alcohol absent {Parents: Mild punishments ! Mild anxiety Alcohol present {Parents: Severe punishments ! Strong anxiety Although the child typically feels a mild amount of anxiety around their parents, the sight or smell of alcohol in the presence of their parents greatly increases the child’s anxiety. Thus: Alcohol absent {Parents ! Mild anxiety Alcohol present {Parents ! Strong anxiety The conditioned response of anxiety to the parents is intensified by the presence of alcohol. The alcohol, the sight or smell of which by itself doesn’t bother the child, is therefore an occasion setter that increases the amount of anxiety the child feels in the presence of the parents. Note that we will discuss the risks of using punishment, more generally, in Chapter 9. This scenario provides one example of the risks of using punishment, as it can damage relationships. Because the real world consists of a complex mixture of stimuli, occasion setting is an important factor in many instances of classical conditioning. A dog will be more excited when their owner puts on shoes at about the time that they typically go for a walk than when their owner puts on shoes at any other time of the day. And hikers are more anxious around bears with cubs than they are around bears without cubs. The additional stimuli present in these circumstances (walk time and bear cubs) indicate a higher probability of certain events (walk time and bear attack). 157 158 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Latent Inhibition Do we condition more readily to stimuli that are familiar or unfamiliar? You might think that familiar stimuli are more readily conditioned: if we already know something about an object or topic, it should be easier to learn more about it. In fact, in what is known as latent inhibition, a familiar stimulus is more difficult to condition as a CS than an unfamiliar (novel) stimulus.3 Or, stated the other way around, an unfamiliar stimulus is more readily conditioned as a CS than a familiar stimulus. For example, if a dog has often heard the sound of a metronome prior to it being used in a conditioning experiment, then a standard number of conditioning trials might result in little or no conditioning to the metronome (refer to Figure!4.8). In diagram form: Figure 4.8 In latent inhibition, familiar stimuli are more difficult to condition as CSs than novel stimuli. If a dog has, on many occasions, heard the sound of a metronome prior to conditioning being implemented, then it will be difficult to obtain conditioning to the metronome using a standard number of conditioning trials. (Source: Nairne, 2000.) Metronome No salivation 40 presentations of the metronome NS Metronome Food NS Metronome US Salivation UR No salivation NS 3 Latent inhibition is also known as the CS pre-exposure effect. A related phenomenon, known as the US pre-exposure effect, holds that conditioning is slower with familiar, as opposed to unfamiliar, USs. 4.3 Four Examples of Specificity in Classical Conditioning (Step 1: Stimulus pre-exposure phase in which a metronome is repeatedly presented alone) Metronome (NS) (40 presentations) (Step 2: Conditioning trials in which the pre-exposed metronome is now paired with food) Metronome (NS): Food (US) ! Salivation (UR) (10 trials) (Step 3: Test trial to determine if conditioning has occurred to the metronome) Metronome (NS) ! No salivation (—) If the dog had not been pre-exposed to the metronome and the metronome had instead been a novel stimulus, then the 10 conditioning trials would have resulted in significant conditioning to the metronome. Because of the pre-exposure, however, no conditioning occurred. It will take many more pairings of metronome and food before the metronome will reliably elicit salivation. In the same way that habituation prevents the occurrence of unlearned reflexive responses to redundant stimuli (e.g., a startle response to a repeatedly occurring noise), latent inhibition prevents the development of conditioned responses to redundant stimuli that are coincidentally paired with a US. Such stimuli are likely to be inconsequential with respect to the conditioning event. For example, if a rabbit in a grassy field is attacked by a coyote and then escapes, it will be much more adaptive for the rabbit to associate the attack with the novel scent of the coyote than with the familiar scent of grass. The scent of the coyote is a good predictor of a possible attack, and a fear of that scent will help the rabbit avoid such attacks in the future. A fear of the grass, however, will be completely maladaptive because the rabbit is surrounded by grass every day. It is the novel stimuli preceding the presentation of a US that are most likely to be meaningfully related to it. Difficulties with latent inhibition (as well as habituation) are evident in people who have schizophrenia (Lubow & Gewirtz, 1995). These individuals often have great difficulty attending to relevant stimuli in their environment and are instead distracted by irrelevant stimuli, such as various background noises or people passing nearby. Experiments have revealed that people with schizophrenia display less latent inhibition than is normal—that is, they condition more easily to familiar stimuli—indicating that the disorder partly involves an inability to screen out redundant stimuli. Experiments have also revealed that drugs used to treat schizophrenia tend to increase levels of latent inhibition, thereby normalizing the person’s attentional processes. 159 160 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities Advice for the Lovelorn Dear Dr. Dee, I have two friends who I think would have a wonderful relationship, if only they would think of each other in a romantic way. We all grew up together, so they know each other very well. They complete each other’s sentences, they like the same things, and they’re both very attractive. I’ve mentioned this to each of them, separately, but they laughed at me. They both said that they think of the other like family and couldn’t imagine dating! Why won’t they realize that they’re perfect for each other? Frustrated Matchmaker Dear Matchmaker, On the one hand, your friends are the best judges of who they’re attracted to. On the other hand, their increased familiarity with each other might also mean that they can’t easily develop an attraction or romantic feelings because their prior learning gets in the way. This is in keeping with the process of latent inhibition, in which we condition less readily to familiar stimuli than to unfamiliar stimuli. You might think that it is obvious that these two should be together, romantically, but they may not be able to see what you see. On a positive note, it sounds like they will have a long-lasting relationship—even if it is just as friends. Quick Quiz H Behaviorally yours, 1. In classical conditioning, o s is a procedure in which a stimulus signals whether a CS is likely to be followed by . This stimulus is called a(n) the . 2. In , a familiar stimulus is more difficult to condition as a CS than is an unfamiliar stimulus. 3. Latent inhibition (prevents/promotes) the development of conditioned associations to redundant stimuli. 4. Feeling comfortable around dogs only if they are on leashes is an example of ; not being afraid of guns (even though you accidentally shot yourself in the foot) because you’re a long-time . gun owner is an example of Study Questions Warning In this chapter, you have been exposed to several conditioning procedures. Some are quite similar (such as overshadowing and blocking). Be sure to overlearn these procedures, as students often confuse them, especially under the stress of an examination. In Review To create a summary for this chapter, include a paragraph about acquisition and extinction and another paragraph about how factors like context and personality can affect learning processes. Another paragraph should focus on generalization and discrimination. You’ll likely need at least two additional paragraphs to summarize the variations on classical conditioning and to explain how we now know that contiguity alone is not sufficient for acquisition to occur. Other ways to test yourself on concepts from this chapter include creating diagrams of the various versions of classical conditioning and creating examples of each from your own experiences. Can you think of situations where you were more affected by one cue than another? Or are particular stimuli more noticeable in some situations than others? Try to diagram those examples. Further Exploration As later discussed in Chapter 8, animals’ ability to make fine discriminations has important applications. For example, highly trained “sniffer dogs” are important for finding survivors after an avalanche and are also used by military and police forces for detecting explosives or contraband. Some dogs (scent hounds, in particular) are better at the job, but any dog with a functional nose can be trained with a discrimination procedure: www.akc. org/expert-advice/training/how-to-teach-your-dog-scent-work. The concept of overshadowing has emerged as an important topic in medicine, where diagnosis can be affected by a physician’s attention toward the most salient symptom, leading to critical errors: https://psnet.ahrq.gov/web-mm/diagnostic-overshadowing-dangers. Study Questions 1. Draw a graph of a typical acquisition curve (remember to properly label each axis) and indicate the asymptote of conditioning. 161 162 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities 2. Define the processes of extinction and spontaneous recovery. What typically happens to spontaneous recovery with repeated sessions of extinction? 3. Define disinhibition and give an example. How does disinhibition differ from dishabituation (as covered in Chapter 3)? 4. Describe stimulus generalization and semantic generalization. Give an example of each for a fear conditioning procedure in which the spoken word “rat” was paired with shock, and the stimulus later presented was the spoken word of either “mouse” or “hat.” 5. Describe stimulus discrimination and give an example. Diagram an example of a discrimination training procedure. (When asked to diagram a conditioning procedure like this, be sure to label each component with the appropriate abbreviations, e.g., CS, and use the appropriate symbols of a colon or arrow). 6. Define experimental neurosis and describe Shenger-Krestovnikova’s procedure for producing it. 7. Define higher-order conditioning and sensory preconditioning. Note how they are similar to and different from each other. Diagram an example of each. 8. Define and diagram an example of US revaluation. 9. Define overshadowing and blocking. Note how they are similar to and different from each other. Diagram an example of each. 10. Define occasion setting and latent inhibition. Diagram an example of each. Concept Review acquisition. The process of developing and strengthening a conditioned response through repeated pairings of an NS with a US. asymptote of conditioning. The maximum amount of conditioning that can take place in a particular situation. blocking. The phenomenon whereby the presence of an established CS during conditioning interferes with conditioning of a new CS. compound stimulus. A complex stimulus that consists of the simultaneous presentation of two or more individual stimuli. disinhibition. The sudden recovery of a conditioned response during an extinction procedure when a novel stimulus is introduced. experimental neurosis. An experimentally produced disorder in which animals exposed to unpredictable events develop neurotic-like symptoms. extinction. In classical conditioning, the process whereby a CR can be weakened or eliminated when the CS is repeatedly presented in the absence of the US; also, the procedure whereby this happens, namely, repeatedly presenting the CS in the absence of the US. higher-order conditioning. The process whereby an NS that is associated with a CS (rather than a US) also becomes a CS. Chapter Test latent inhibition. The phenomenon whereby a familiar stimulus is more difficult to condition as a CS than is an unfamiliar (novel) stimulus. occasion setting. A procedure in which a stimulus (known as an occasion setter) signals whether a stimulus is likely to be followed by a US and thereby determines whether that stimulus will act as a CS that elicits a CR. overshadowing. The phenomenon whereby the more salient member of a compound stimulus is more readily conditioned as a CS and thereby interferes with conditioning of the less salient member. semantic generalization. The generalization of a conditioned response to verbal stimuli that are similar in meaning to the CS. sensory preconditioning. When one stimulus is conditioned as a CS, another stimulus with which it was previously associated can also become a CS. spontaneous recovery. The reappearance of a conditioned response to a CS following a rest period after extinction. stimulus discrimination. The tendency for a response to be elicited more by one stimulus than another. stimulus generalization. The tendency for a CR to occur in the presence of a stimulus that is similar to the CS. US revaluation. A process that involves the postconditioning presentation of the US by itself at a different level of intensity, thereby altering the strength of response to the previously conditioned CS. Chapter Test 12. In higher-order conditioning, the CS2 generally elicits a (stronger/weaker) response than does the CS1. 5. The fact that you learned to fear wasps and hornets, as well as bees, after being stung by a bee is an example of stimulus . On the other hand, if you fear only striped snakes and not solid-colored snakes, that would be an example of stimulus . 8. Pavlov regarded spontaneous recovery and disinhibition as evidence that extinction involves the (unlearning/inhibition) of the learned response. 18. While playing tennis one day, you suffer a minor ankle sprain. Two weeks later, you severely twist your ankle while stepping off a curb. You now find yourself very worried about playing tennis. This is an example of . 23. According to Eysenck, psychopaths tend to be extreme (extroverts/introverts) who condition (easily/poorly). 20. Midori feels anxious whenever the manager walks into the store accompanied by the owner because the manager always finds fault with the employees when the owner is there. This is best seen as an example of , with the owner functioning as the . 14. Two examples of specificity in conditioning, known as and , involve pairing a compound stimulus with a US. They both provide evidence that contiguity between the NS and the US (is/is not) a sufficient factor in conditioning. 163 164 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities 2. Following an experience in which you were stung by a bee and subsequently developed a fear of bees, you are hired for a one-day job in which your task is to catch bees for a biologist. During the day, you do not get stung. As a result, your fear of bees will likely (decrease/increase), a process known as ____________________. 10. The researcher feels that you have done such a fine job catching bees that she hires you for another day. At the start of the next day, you will likely find that your fear of bees has (completely disappeared/partially returned), a phenomenon known as ____________________. 22. By the end of the second day, your fear of bees has mostly disappeared. However, you see an owl sitting in a tree, which is extremely unusual in that part of the country. You decide to catch one more bee but find that your fear of bees is now somewhat stronger. The sudden recovery of your fear response to bees is an example of a process known as ____________________. 15. Marty once played in an all-star game alongside Bobby Orr (a famous and talented professional hockey player). Marty scored two goals and an assist, as did Orr. Orr was later voted the game’s most valuable player, while Marty’s name was barely mentioned. This situation seems analogous to the ____________________ effect in classical conditioning. 25. Imagine that Pavlov himself learned to salivate to the sound of a bell after watching a dog being trained to salivate to a bell. This situation might have arisen if the dogs were being fed bites of juicy steak, the sight of which is, for many people, a (CS1/CS2) for salivation. If so, the bell would then become a (CS1/CS2) for salivation through its association with the sight of the steak. Thus, of the different types of extensions to classical conditioning, this is most similar to the process of ____________________. 19. If one thinks of a conditioned stimulus as being a particularly salient stimulus, then a blocking procedure is very similar to a(n) ____________________ procedure. 3. Consider the following example in which the sight of different people comes to elicit feelings of anger: (Step 1) Seeing Jaden: Jaden’s rude behavior ! Anger toward Jaden Seeing Jaden ! Anger toward Jaden (Step 2) Seeing Amir: Seeing Jaden ! Anger toward Jaden Seeing Amir ! Anger toward Amir This is an example of ____________________ conditioning. 6. The procedure of extinction involves the _______________________________, while the process of extinction involves the _____________________________. 11. In higher-order conditioning, conditioning of the CS 1 is sometimes called ____________________ conditioning, and conditioning of the CS2 is called ____________________ conditioning. 24. The gradual strengthening of a classically conditioned fear response by repeated pairings of a tone with a shock is an example of the process of ____________________. During this process, the early pairings of tone and shock are likely to produce (larger/smaller) increases in conditioning than the later pairings. Answers to Chapter Test 1. The maximum amount of conditioning that can take place in a particular situation is known as the ____________________ of conditioning. 9. Consider the following example in which feelings of tension toward your former friend, Adar, are also elicited by a restaurant where you often used to meet up with him: (Step 1: Repeated experiences in a restaurant) Restaurant: Adar (Step 2: Not in a restaurant) Adar: Argument ! Tension Adar ! Tension (Step 3) Restaurant ! Tension This process is best interpreted as an example of ____________________. 4. Based partially on Pavlov’s work on experimental neurosis, Eysenck concluded that people who are ____________________ tend to be highly reactive to external stimulation, condition easily, and develop anxiety-type symptoms in reaction to stress. By contrast, people who are ____________________ tend to be less reactive, condition less easily, and develop physical-type symptoms in reaction to stress. 17. You once played in an all-star game alongside Antonio, an unknown basketball player (like you). Antonio, however, is very tall and quite noticeable on the court. Although you both played equally well, almost all the credit for the win went to ____________________, which seems analogous to the ____________________ effect in classical conditioning. 13. If the scent of ammonia and the ticking of a clock are combined to form a compound stimulus, then the two stimuli are being presented (simultaneously/ successively). 21. Rasheed had never experienced a more difficult multiple-choice test. Virtually every alternative for every question looked equally correct. By the end of the exam, he felt extremely anxious. Rasheed’s experience is somewhat analogous to a phenomenon discovered by Pavlov’s associates, which they called ____________________. 16. A student has great difficulty ignoring irrelevant material that is sometimes mentioned during a lecture and is easily distracted. This student likely also displays (stronger/weaker) evidence of ____________________ inhibition compared to an average student. 7. A person who fears dogs also feels anxious when they hear the word “canine.” This is an example of ____________________. Answers to Chapter Test 1. asymptote 2. decrease; extinction 3. higher-order 4. introverts; extroverts 5. generalization; discrimination 6. repeated presentations of the CS without the US; resultant decrease in the strength of the conditioned response 7. semantic generalization 165 166 Chapter 4 Classical Conditioning: Basic Phenomena and Various Complexities 8. inhibition 9. sensory preconditioning 10. partially returned; spontaneous recovery 11. first-order; second-order 12. weaker 13. simultaneously 14. blocking; overshadowing; is not 15. blocking (with Orr being analogous to an established CS) 16. weaker; latent 17. Antonio; overshadowing 18. US revaluation (or US inflation) 19. overshadowing 20. occasion setting; occasion setter 21. experimental neurosis 22. disinhibition 23. extroverts; poorly 24. acquisition; larger 25. CS1; CS2; higher-order conditioning Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications Estella thought Juan looked a bit tipsy as he left the picnic to drive home. She wondered if she should tell him that the supposedly nonalcoholic punch he had been drinking was actually spiked with vodka. On the other hand, he had only had one glass. Surely, he couldn’t be drunk. Chapter Outline 5.1 Underlying Processes in Classical Conditioning Stimulus-Substitution Theory Preparatory-Response Theory (and the CompensatoryResponse Model) Rescorla–Wagner Theory 5.2 Practical Applications of Classical Conditioning Understanding Phobias Treating Phobias Aversion Therapy for Problem Behaviors Medical Applications of Classical Conditioning Learning Objectives 5-1 Distinguish between stimulus-substitution theory and preparatory response theory, including empirical support for each. 5-2 Describe the compensatory-response model of conditioning and its application to drug tolerance and overdose. 5-3 Describe the Rescorla–Wagner theory of conditioning, including how that theory accounts for phenomena like overshadowing and blocking. 5-4 Describe and diagram various ways that fears and phobias can be acquired through conditioning. 5-5 Explain the role of temperament, preparedness, and prior experiences in fear conditioning. 5-6 Distinguish among various methods to reduce fear or treat phobias. 5-7 Describe aversive conditioning treatment procedures as well as medical applications of classical conditioning. 167 168 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications 5.1 Underlying Processes in Classical Conditioning By now, you have probably realized that classical conditioning is not as simple a process as it first seems. It is a complex phenomenon that is only slowly yielding its secrets to researchers. The following sections discuss major theoretical notions concerning the underlying processes of classical conditioning. You will learn how some of these theories have resulted in findings of practical importance. Stimulus-Substitution Theory We begin with Pavlov (1927), who viewed classical conditioning as largely a matter of replacing one stimulus with another. Thus, according to Pavlov’s stimulus-substitution theory, the CS acts as a substitute for the US. For example, pairing a tone with food results in the tone becoming a substitute for the food; the outcome is that the tone can now elicit salivation. And pairing the sound of a click with an air puff to the eye results in the click becoming a substitute for the air puff; the outcome is that the click now elicits an eyeblink. Pavlov was a physiologist who believed that classical conditioning was an indirect way of studying neurological processes in the brain. Thus, he often made inferences about the kinds of neurological processes that are activated during conditioning. He claimed that presentation of a US, such as food, activates an area of the cerebral cortex (the outermost layer of the brain) that is responsible for sensing the occurrence of that event. Activation of this “food center” in the brain in turn activates another part of the cortex (the “salivation center”) that produces the unconditioned response of salivation. Food ! Activates food center in cortex ! Activates salivation center in cortex!! Salivation Pavlov also believed that the presentation of a neutral stimulus, such as a light, activates another area of the cortex responsible for detecting that type of stimulus. According to Pavlov, when the light is presented just before the food, a connection is formed between the area of the cortex activated by the light and the area activated by the food. As a result, activation of the light center of the cortex also activates the food center of the cortex, which in turn produces salivation. In other words, Pavlov believed that the presentation of the light set in motion the following sequence of events: Light ! Activates light center in cortex ! Activates food center in cortex ! Activates salivation center in cortex ! Salivation Pavlov’s notions about the kinds of neurological processes underlying classical conditioning are now considered to be incorrect. The actual processes involved are known to be considerably more complex than he presumed. 5.1 Underlying Processes in Classical Conditioning Nevertheless, this does not negate all aspects of Pavlov’s theory. For example, consider the notion that the conditioned stimulus (CS) is somehow a direct substitute for the US. In at least some cases, it seems as though animals do react to the CS as if it were the US. The dog salivates to the light just as it does to food. More importantly, the dog may even approach the light and start to lick it, as though pairing the light with the food resulted in the light being perceived as edible (Pavlov, 1941). This sort of phenomenon, now known as sign tracking, is discussed more fully in Chapter 12. A particular difficulty for stimulus-substitution theory is that if the CS is acting as a substitute for the US, then the CR should always be the same, or at least highly similar, to the UR. Although this is often the case, it sometimes is not the case; in fact, there are sometimes substantial differences between the CR and the UR. For example, a rat that receives a foot shock (the US) will probably jump (the UR). However, if it sees a light (CS) that has been paired with a foot shock, it will often freeze (the CR). Why would the rat jump in one instance and freeze in the other? An examination of the rat’s natural response to danger gives us a clue. If a rat is attacked by a snake, jumping straight up (and rats can really jump!) may cause the snake to miss. On the other hand, if a rat detects a snake in the vicinity, tensing its muscles and freezing will minimize the possibility of being detected or, if the rat is attacked, will enable it to jump quickly. This suggests that the purpose of the CR, rather than merely being a version of the UR, is to ready the organism for the occurrence of the US—which brings us to our next theory of conditioning. Preparatory-Response Theory (and the Compensatory-Response Model) According to the preparatory-response theory of conditioning, the purpose of the CR is to prepare the organism for the presentation of the US (Kimble, 1961, 1967). The dog salivates to the tone to get ready for food, and the rat freezes in response to the light to get ready for the shock. Note that in one case, the preparatory response is quite similar to the UR, whereas in the other case it is quite different. Thus, a major advantage of preparatory-response theory is that, unlike stimulus-substitution theory, it accounts for situations in which the CR and the UR are sometimes different. An interesting example of conditioned preparatory-responses involves cases in which the CR seems to be the exact opposite of the original UR. This often occurs with drug reactions, so we will illustrate it using the example of heroin. Heroin is an opiate drug, in the same class as morphine, oxycodone, and fentanyl. It is an analgesic and a narcotic, meaning that it kills pain and makes you sleepy. It is highly addictive, and users report that the high associated with heroin is very powerful (Strang et al., 2020). Imagine that someone always injects heroin in the presence of certain environmental cues, such as in a particular room. Heroin has several effects 169 170 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications on the body, but we will focus on just one of them for now, which is a decrease in blood pressure (represented by “BP” in the following diagrams). Thus, regularly injecting heroin in a particular setting involves the following sequence of events: Heroin cues: (NS) Heroin (US) ! BP decreases (UR) If this were a normal conditioning procedure, one might expect that the heroin-related cues would eventually become a CS that will itself elicit a decrease in blood pressure. But in reality, the opposite occurs. With repeated drug use, the presence of the heroin-related cues elicits not a decrease in blood pressure, but an increase in blood pressure! Heroin cues: (CS) ! BP Increases (CR) How can this be? Remember the opponent-process theory of emotion that we learned about in Chapter 3. Recall how certain stimuli can elicit both a primary response (the a-process) and a compensatory response (the b-process). Thus, according to the compensatory-response model, a CS that has been repeatedly associated with the primary response (the a-process) to a US will eventually come to elicit a compensatory response (the b-process). To help clarify this, let us examine the heroin example in more detail. Repeatedly injecting heroin does not simply elicit a response, but instead sets in motion a chain of events. The heroin directly elicits an immediate decrease in blood pressure (the a-process) that in turn elicits a compensatory increase in blood pressure (the b-process). Heroin ! BP decreases (a-process) ! BP Increases (b-process) In terms of stimuli and responses, the heroin is a US that naturally elicits a decrease in blood pressure, and the decrease in blood pressure is itself a US that naturally elicits an increase in blood pressure. Therefore, the decrease in blood pressure is both an unconditioned response (UR) to heroin and an unconditioned stimulus (US) that elicits a compensatory increase in blood pressure. Heroin (US) ! BP decreases (UR/US) ! BP Increases (UR) Notice that there are two USs in this sequence, with which the cues in the environment could potentially become associated: one US is the heroin and the other is the decrease in blood pressure that results from the heroin. What happens in compensatory conditioning is that the heroin-related cues, such as being in a certain room, become associated not with the heroin itself but with the primary response to heroin—that is, with the decrease in blood pressure. As a result, these cues eventually come to elicit the compensatory reaction to that response. So the actual conditioning that takes place with heroin is as follows: Heroin cues (NS): BP decreases (US) ! BP Increases (UR) Heroin cues (CS) ! BP Increases (CR) 5.1 Underlying Processes in Classical Conditioning 171 Why would this type of compensatory conditioning occur? Remember how in the opponent-process theory, the compensatory reactions to a US serve to maintain a state of homeostasis (internal balance). If these compensatory reactions start occurring before the US is presented, they will be even more effective in minimizing the disturbance produced by the US. For example, if the compensatory reaction to the heroin (an increase in blood pressure) can be elicited just before the injection of heroin, then the immediate physical reaction to the heroin (the decrease in blood pressure) will be effectively moderated. In this sense, a conditioned compensatory response allows the body to prepare itself ahead of time for the onslaught of the drug. Conditioned compensatory responses therefore constitute an extreme form of preparatory response to certain environmental events. 2. 3. 4. 5. theory, the CS acts as a substitute for the US. According to this theory, the CR and UR should always be the same or at least highly similar. As it turns out, this is (true/false). According to theory, the purpose of the CR is to prepare the organism for the occurrence of the US. According to the model of drug conditioning, a CS that has been paired with a drug will eventually come to reaction to the drug. elicit a c More precisely, the CS has become associated with the (a-process/b-process) being elicited by the drug; the CS therefore eventually comes to elicit the (a-process/b-process) that was originally elicited by the (a-process/b-process). Diagram the actual events involved in the conditioning of an increase in blood pressure in response to a needle that has been consistently associated with heroin administration (hint: the US in the conditioning of this response is not heroin): Needle (NS): (US) ! Needle (CS) ! (CR) (UR) 6. Shock naturally elicits an increase in heart rate. In this case, shock is the (NS/CS/ US) and the increase in heart rate is the (CR/UR). 7. Following from question 6, an increase in heart rate naturally elicits a compensatory decrease in heart rate. For this sequence of events, the increase in heart rate is the (NS/CS/US) and the decrease in heart rate is the (CR/UR). 8. Following from question 6, a tone that is repeatedly paired with shock will eventually come to elicit a compensatory decrease in heart rate. Diagram the actual events involved in this type of conditioning (pay particular attention to what the US actually consists of). Tone (NS): Tone (CS) ! (US) ! (UR) (CR) The compensatory-response model obviously has important implications for drug addiction and substance use disorders. Drug addictions are partly Quick Quiz A 1. According to 172 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications motivated by a tendency to avoid the symptoms of withdrawal, which are essentially the compensatory responses to the effect of the drug. For example, heroin produces a decrease in blood pressure as well as a combination of other effects, which the drug user experiences as pleasant feelings of relaxation and euphoria. This relaxing effect of heroin in turn elicits compensatory reactions that, on their own, would be experienced as unpleasant feelings of agitation. Heroin use therefore results in the following process of conditioning: Heroin cues (NS): Relaxing effect of ! Compensatory agitation (UR) heroin (US) These conditioned feeling of agitation are perceived as cravings or symptoms of withdrawal. It should be noted that for some drugs, like alcohol, conditioning of compensatory responses requires repeated use. For opiate drugs, however, a single trial can result in significant compensatory conditioning (Strang et al., 2020). The presence of drug-related cues is one of the strongest reasons why people continue to battle cravings long after they have stopped using a drug. Think of an individual who always uses heroin in a particular environment, goes into a rehab program, and then returns home to their usual environment. When they return to the environment in which they had previously used heroin, they will very likely become tense and agitated, which they will interpret as withdrawal symptoms and a craving for heroin. To escape from these symptoms, they will be sorely tempted to once more take heroin. To the extent that withdrawal symptoms are elicited by cues associated with drug use, then removing those cues should weaken the withdrawal symptoms and make it easier to remain abstinent. This possibility is supported by anecdotal evidence. Many American soldiers used heroin during their tour of duty in Vietnam, leading to fears that they would remain addicted when they returned home. These fears, however, did not materialize (Robins, 1974). One explanation for this is that the drastic change in environment when the soldiers returned home removed many of the cues associated with heroin use, thereby alleviating the symptoms of withdrawal and making it easier for them to remain heroin-free. Unfortunately, for many people trying to kick a habit, whether it is alcohol, cigarettes, or heroin, it is often not possible to completely avoid all cues associated with the drug. For this reason, modern treatments for drug addiction often include procedures designed to extinguish the power of such cues. For example, someone attempting to quit smoking may be required to remain in the presence of cigarettes for a long period of time without smoking. Repeated presentations of the CS (the sight of the cigarettes) in the absence of the US (nicotine ingestion) should result in weaker and weaker CRs (cravings for a smoke). Of course, this process can initially be very difficult—and in the case of severe alcoholism, even dangerous due to the severity of withdrawal symptoms. It therefore requires careful management; however, once accomplished, it can significantly reduce the possibility of a relapse. (Refer also to Sokolowska, Siegel, & Kim, 2002 for a discussion of how some CSs can be internal, such as feelings of stress that lead to smoking, and how the effect of these internal cues may also need to be extinguished.) 5.1 Underlying Processes in Classical Conditioning The compensatory-response model also has implications for drug tolerance (Siegel, 1983, 2005). For example, if you have a habit of always drinking in a particular setting, then the various cues in that setting—people greeting you as you walk in the front door of the bar; the stool you always sit on— become CSs for the effect of alcohol. The presence of these CSs will initiate physiological reactions that compensate for the alcohol you are about to consume. As a result, in the presence of these CSs, you should have greater tolerance for alcohol than you would in their absence. Research has confirmed this association. In a study by McCusker and Brown (1990), participants consumed alcohol in either an “alcohol expected” environment (i.e., alcohol was consumed in a simulated lounge during the evening with pub noises playing in the background) or an “alcohol unexpected” environment (i.e., alcohol was consumed during the day in an office environment). Those who consumed alcohol in the expected environment performed significantly better on various measures of cognitive and motor functioning compared to those who consumed alcohol in the unexpected environment. They also showed smaller increases in pulse rate. This suggests that the alcohol-related cues in the expected condition (evening, lounge setting) elicited compensatory reactions that partially compensated for the effects of the alcohol (refer also to Bennett & Samson, 1991). Conversely, this also means that if you consume alcohol in an environment where you typically do not drink (e.g., a business lunch), the alcohol could have a much stronger effect on you than if you consumed it in an environment where you typically do drink (e.g., your local bar). This means that your ability to drive safely could be significantly more impaired following a lunchtime martini than after an evening drink at a familiar bar. Worse yet, even if you do consume the drink at a familiar bar, consider what happens when you leave that setting. Your compensatory reactions might be significantly reduced because you have now removed yourself from the alcohol-related cues that elicited those reactions. As a result, you could become more intoxicated during the drive home from the bar than you were in the bar (Linnoila, Stapleton, Lister, Guthrie, & Eckhardt, 1986). This means that the amount of alcohol you consume is not, by itself, a reliable gauge for determining how intoxicated you are. (Thus, going back to the opening vignette for this chapter, why should Estella be concerned about Juan’s ability to drive?)1 It should be noted that there are exceptions to the typical compensatory reactions to a CS. Stimuli associated with drug use sometimes elicit druglike reactions rather than drug-compensatory reactions. In other words, the stimuli become associated with the primary response to the drug rather than the compensatory response. For example, in one study, rats became more sensitive to cocaine when it was administered in the usual cocaine 1 The type of alcohol consumed can also have an effect. People become significantly more intoxicated following consumption of an unusual drink (such as a strange liqueur) rather than a familiar drink (such as beer). The familiar drink can be seen as a CS for alcohol that elicits compensatory reactions to the alcohol (Remington, Roberts, & Glautier, 1997). 173 174 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications administration environment than in a different one (Hinson & Poulos, 1981). The CSs for cocaine administration apparently elicited reactions that mimicked the drug, thereby strengthening its effect. There is also evidence that stimuli associated with drug use sometimes elicit both compensatory responses in one system of the body and drug-like responses in another. For example, a cup of decaffeinated coffee (which, appearing as coffee, is normally a cue for caffeine consumption) can produce a caffeine-like increase in alertness and a caffeine-compensatory decrease in salivation (Rozen, Reff, And Furthermore Conditioned Compensatory Responses and Drug Overdose The compensatory-response model has also been used to explain incidents of drug overdose. Many “overdose” fatalities do not, in fact, involve an unusually large amount of the drug. For example, heroin users often die after injecting a dosage that has been well tolerated on previous occasions. A critical factor appears to be the setting within which the drug is administered. As we have seen, if a heroin user typically administers the drug in the presence of certain cues, those cues become CSs that elicit compensatory reactions to the drug. Tolerance to heroin therefore is much greater in the presence of those cues than in their absence. Anecdotal evidence supports this possibility. Siegel (1984) interviewed 10 survivors of heroin overdose, 7 of whom reported that the overdose had been preceded by an unusual change in the setting or drug administration procedure. For example, one woman reported that she overdosed after hitting a vein on the first try at injecting the drug, whereas she usually required several tries. Further evidence comes from studies with rats that had become addicted to heroin. When the cues usually associated with heroin were absent, the rats’ ability to tolerate a large dose was markedly reduced to the point that many of the rats died. Thus, heroin-tolerant rats who were administered a very strong dose of heroin in a novel (non-heroin related) setting were more likely to die than those who received the dose in the setting previously associated with the drug (Siegel, Hinson, Krank, & McCully, 1982). Siegel (1989) describes additional cases that illustrate the dangers of drug overdose resulting from conditioning effects. For example: The respondent (E.!C.) was a heavy user of heroin for three years. She usually self-administered her first, daily dose of heroin in the bathroom of her apartment, where she lived with her mother. Typically, E.!C. would awake earlier than her mother, turn on the water in the bathroom (pretending to take a shower), and self-inject without arousing suspicion. However, on the occasion of the overdose, her mother was already awake when E.!C. started her injection ritual, and knocked loudly on the bathroom door telling E.!C. to hurry. When E.!C. then injected the heroin, she immediately found that she could not breathe. She was unable to call her mother for help (her mother eventually broke down the bathroom door and rushed E.!C. to the hospital, where she was successfully treated for heroin overdose). (pp. 155–156) Siegel goes on to explain that the mother knocking on the bathroom door was an unusual cue that may have disrupted the environmental CSs that would normally have elicited compensatory reactions to the heroin. 5.1 Underlying Processes in Classical Conditioning 175 Mark, & Schull, 1984; refer also to Eikelboom & Stewart, 1982; Lang, Ross, & Glover, 1967). Thus, the circumstances in which conditioning results in drug-like reactions versus drug-compensatory reactions are complex and not entirely understood (Siegel, 1989). (Refer also to “Conditioned Compensatory Responses and Drug Overdose” in the And Furthermore box.) 2. 3. 4. 5. of withdrawal are likely to be (stronger/weaker) in the presence of drug-related cues. This is because the drug-related cues tend to elicit (primary/compensatory) responses to the drug that are experienced as cravings. In keeping with the compensatory-response model, modern treatments for drug addiction often recommend (exposure to/removal of) drug-related cues to allow (conditioning/extinction) of the cravings to take place. We tend to have (higher/lower) tolerance for a drug in the presence of cues associated with taking the drug. Suppose someone always injects heroin in their bedroom at home, but one time stays overnight at a friend’s house and decides to use there. They will likely experience a(n) (increased/decreased) level of intoxication to the drug at the friend’s house. A person who drinks a glass of wine in a fine restaurant is likely to be (more/ less) affected by the alcohol than if they drank the same amount of wine in a courtroom. Rescorla–Wagner Theory One of the most influential theories of classical conditioning was proposed by Rescorla and Wagner (1972). Their theory attempted to explain the effect of each conditioning trial on the strength, or what might be called the “associative value,” of the CS in its relationship to the US. The Rescorla– Wagner theory proposes that a given US can support only so much conditioning, and this amount of conditioning must be distributed among the various CSs that are present. Another way of saying this is that there is only so much associative value available to be distributed among the cues associated with the US. One assumption of this theory is that stronger stimuli (USs and CSs) support more conditioning than do weaker stimuli. For example, the use of a highly preferred food as the US produces a stronger conditioned response of salivation than does a less preferred food. Imagine, for example, that a tone paired with a highly preferred food (say, steak) elicits a maximum of 10 drops of saliva, while a tone paired with a much less preferred food (say, dog food) elicits only 5 drops of saliva. If we regard each drop of saliva as a unit of associative value, then we could say that the highly preferred food supports a maximum associative value of 10 units, while the less preferred food supports a maximum associative value of 5 units. We can use the following format to diagram the changes in associative value (we will assume the highly preferred food is the US): Tone (V 5 0): Food (Max 5 10) ! Salivation Tone (V 5 10) ! Salivation Quick Quiz B 1. According to the compensatory-response model of drug addiction, symptoms 176 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications The letter V will stand for the associative value of the CS (which at the start of conditioning is 0). The term Max will stand for the maximum associative value that can be supported by the US once conditioning is complete. In our example, imagine V as the number of drops of saliva the tone elicits—0 drops of saliva to begin with and 10 drops once the tone is fully associated with the food—and Max as the maximum number of drops of saliva that the tone can potentially elicit if it is fully associated with the food. (If this is starting to look a bit mathematical to you, you are correct. In fact, the model can be expressed in the form of an equation. For our purposes, however, the equation is unnecessary.)2 Now suppose that a compound stimulus consisting of a tone and a light are repeatedly paired with the food, to the point that the compound stimulus obtains the maximum associative value. [Tone 1 Light] (V 5 0): Food (Max 5 10) ! Salivation [Tone 1 Light] (V 5 10) ! Salivation This associative value, however, must somehow be distributed between the two component members of the compound. For example, if the tone is slightly more salient than the light, then the tone might have picked up six units of associative value while the light picked up only four units. In other words, when tested separately, the tone elicits six drops of saliva while the light elicits four. Tone (V 5 6) ! Salivation Light (V 5 4) ! Salivation If the tone were even more salient than the light—for example, it was a very loud tone and a very faint light—then overshadowing might occur, with the tone picking up nine units of associative value and the light only one unit: [Loud tone 1 Faint light] (V 5 0): Food (Max 5 10) ! Salivation Loud tone (V 5 9) ! Salivation Faint light (V 5 1) ! Salivation The loud tone now elicits nine drops of saliva (a strong CR) while the faint light elicits only one drop of saliva (a weak CR). Thus, the Rescorla– Wagner explanation for the overshadowing effect is that there is only so much associative value available (if you will, only so much spit available) for conditioning, and if the stronger stimulus in the compound picks up most or all of the associative value, then there is little or no associative value left over for the weaker stimulus. 2 For those who are mathematically inclined, the equation for the Rescorla–Wagner model is DV 5 k(l – V), where V is the associative value of the CS, l (“lambda”) represents the maximum associative value that the CS can hold (i.e., the asymptote of learning), and k is a constant that represents the salience of the CS and US (with greater salience supporting more conditioning). For more information on the use of this equation, refer to Domjan (2015). 5.1 Underlying Processes in Classical Conditioning As can be seen, the Rescorla–Wagner theory readily explains conditioning situations involving compound stimuli. Take, for example, a blocking procedure. One stimulus is first conditioned to its maximum associative!value: Tone (V 5 0): Food (Max 5 10) ! Salivation Tone (V 5 10) ! Salivation This stimulus is then combined with another stimulus for further conditioning trials: [Tone 1 Light] (V 5 10 1 0 5 10): Food (Max 5 10) ! Salivation But note that the food supports a maximum associative value of only 10 units, and the tone has already acquired that much value. The light can therefore acquire no associative value because all of the available associative value has already been assigned to the tone. Thus, when the two stimuli are later tested for conditioning, the following occurs: Tone (V 5 10) ! Salivation Light (V 5 0) ! No salivation So far we have described the Rescorla–Wagner theory in relation to changes in associative value. The theory can also been interpreted in more cognitive terms. To say that a CS has high associative value is similar to saying that it is a strong predictor of the US, or that the subject strongly “expects” the US whenever it encounters the CS. Thus, in the previous example, to say that the tone has high associative value means that it is a good predictor of food and that the dog “expects” food whenever it hears the tone. In the case of blocking, however, the tone is such a good predictor of food that the light with which it is later paired becomes redundant, and the presence of the light does not affect the subject’s expectations about food. As a result, no conditioning occurs to the light. In general, then, conditioning can be viewed as a matter of building the subject’s expectations that one event will follow another. The Rescorla–Wagner theory also leads to some counterintuitive predictions. Consider what happens if you first condition two CSs to their maximum associative value and then combine them into a compound stimulus for further conditioning. For example, suppose we condition a tone to its maximum associative value, as follows: Tone (V 5 0): Food (Max 5 10) ! Salivation Tone (V 5 10) ! Salivation and then do the same for the light: Light (V 5 0): Food (Max 5 10) ! Salivation Light (V 5 10) ! Salivation We now combine the tone and the light into a compound stimulus and conduct further conditioning trials: [Tone 1 Light] (V 5 10 1 10 5 20): Food (Max 5 10) ! Salivation 177 178 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications Note that the tone and the light together have 20 units of associative value (10 for the tone and 10 for the light). However, the maximum associative value that can be supported by the food at any one moment is only 10 units. This means that the associative value of the compound stimulus must decrease to match the maximum value that can be supported by the US. Thus, according to the Rescorla–Wagner theory, after several pairings of the compound stimulus with food, the total associative value of the compound stimulus will be reduced to 10: [Tone 1 Light] (V 5 10) ! Salivation This in turn means that when each member in the compound is tested separately, its value also will have decreased. For example: Tone (V 5 5) ! Salivation Light (V 5 5) ! Salivation Quick Quiz C Thus, even though the tone and light were subjected to further pairings with the food, the associative value of each decreased (i.e., each stimulus elicited less salivation than it originally did when it had been conditioned individually). This is a counterintuitive result in that one would normally expect further pairings between a CS and US to either maintain or strengthen the conditioning rather than weaken it. The effect we have just described is known as the overexpectation effect, which is the decrease in a CR that occurs when two separately conditioned CSs are combined into a compound stimulus for further pairings with the US. It is as though presenting the two CSs together leads to an “overexpectation” about what will follow. When this expectation is not fulfilled, the subject’s expectations are modified downward. As a result, each CS in the compound loses some of its associative value. Although the Rescorla–Wagner model has been a source of inspiration for researchers, not all of its predictions have been confirmed. As a result, revisions to the model have been proposed along with alternative models. Some behaviorists (e.g., Pierce & Epling, 1995) have also criticized the common practice of interpreting the Rescorla–Wagner model in cognitive terms by arguing that the concept of associative value, which can be objectively measured by the strength of the CR, makes inferences about mentalistic processes unnecessary. Despite these debates, however, few models have been as productive in furthering our understanding of the underlying processes of classical conditioning. 1. The Rescorla–Wagner theory proposes that a given can support only so much conditioning, and this amount of conditioning must be available. distributed among the various 2. In general, stronger USs support (more/less) conditioning than weaker USs, and stronger CSs support (more/less) conditioning than weaker CSs. 3. According to the Rescorla–Wagner theory, overshadowing occurs because the more salient CS picks up (most/little) of the associative value available in that setting. 5.1 Underlying Processes in Classical Conditioning 179 4. According to the Rescorla–Wagner theory, blocking occurs because the (CS/NS/ US) in the compound has already picked up all of the available associative value. 5. Suppose a compound stimulus has an associative value of 25 following conditioning. According to the Rescorla–Wagner theory, if one CS has acquired 15 units of associative value, the other CS must at most have acquired units of associative value. 6. Suppose a tone and a light are each conditioned with food to a maximum associative value of eight units. If the tone and light are combined into a compound stimulus for further conditioning trials, the associative value of each stimulus must necessarily (decrease/increase). This is known as the effect. o Advice for the Lovelorn Dear Dr. Dee, My friend says that if you are deeply and madly in love with someone, then you will necessarily be much less interested in anyone else. I think my friend is wrong. There is no reason why someone can’t be deeply in love with more than one person at a time. So who is right? The Wanderer Dear Wanderer, I honestly don’t know who is right. We don’t usually ask such questions about friendship or about love for our children or other family members, but people tend to have very different views about how much romantic love is possible. Your friend’s hypothesis does seem somewhat consistent with the Rescorla–Wagner theory. If feelings of love are to some extent classically conditioned responses, then the more love you feel for one person (meaning that they are a distinctive CS that has strong associative value), the less love you might feel for alternative partners who are simultaneously available (because there is little associative value left over for those other CSs). In other words, there is only so much love (so much associative value) to go around, and strong romantic feelings for one person will result in weak romantic feelings for others. In keeping with this, you can occasionally encounter people who report being so “in love” with someone—at least in the early stages of a relationship—that they are attracted to no one else. (I remember a movie star once being interviewed who was madly in love. He remarked that he had never thought it possible that he could so completely lose interest in other women.) It is the case, however, that some people have strong romantic feelings for multiple partners. Perhaps those people have a larger maximum associative value for romantic feelings to distribute among different partners or perhaps they are responding to distinct aspects of each partner. Behaviorally yours, 180 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications 5.2 Practical Applications of Classical Conditioning Understanding Phobias A particularly salient way that classical conditioning affects our lives is through its involvement in the development of fears and anxieties. As already noted, a conditioned fear response can be elicited by a previously neutral stimulus that has been associated with an aversive stimulus. In most cases, this sort of fear conditioning is a highly adaptive process because it motivates the individual to avoid a dangerous situation. A person who is bitten by a dog and learns to fear dogs is less likely to be bitten in the future simply because they will tend to avoid dogs. This process, however, occasionally becomes exaggerated, with the result that we become very fearful of events that are not at all dangerous or only minimally dangerous. Such extreme, irrational fear reactions are known as phobias. In many cases, these phobias seem to represent a process of overgeneralization, in which a conditioned fear response to one event has become overgeneralized to other harmless events. Thus, although it may be rational to fear a mean-looking dog that once bit you, it is irrational to fear a friendly-looking dog that has never bitten you. Watson and Rayner’s “Little Albert” The importance of classical conditioning and overgeneralization in the development of phobias was first proposed by John B. Watson and Rosalie Rayner. In 1920, Watson and Rayner published a now-famous article in which they described their attempt to condition a fear response in an 11-month-old infant named Albert. Albert was reported to be a healthy, well-developed child, whose mother worked as a wet nurse in the hospital where the tests were conducted. Albert was described as a “stolid and unemotional” child who almost never cried. In fact, he seemed to display an unusual level of emotional stability. The researchers began the experiment with a baseline session in which they tested Albert’s reactions to a variety of objects. These included a white rat, a rabbit, a dog, some cotton wool, and even a burning newspaper. None of the objects elicited any fear and, in fact, Albert often attempted to handle them. He was, however, startled when the experimenters made a loud noise by banging a steel bar with a hammer. The experimenters thus concluded that the loud noise was an unconditioned stimulus that elicited a fear response (or, more specifically, a startle reaction), whereas the other objects, such as the rat, were neutral stimuli with respect to fear: Loud noise (US) ! Fear (indicated by startle reaction) (UR) Rat (NS) ! No fear (—) 5.2 Practical Applications of Classical Conditioning In the next part of the experiment, Watson and Rayner (1920) paired the white rat (NS) with the loud noise (US). The rat was presented to Albert, and just as his hand touched it, the steel bar was struck with the hammer. In this first conditioning trial, Albert “jumped violently and fell forward, burying his face in the mattress. He did not cry, however” (p. 4). He reacted similarly when the trial was repeated, except that this time he began to whimper. The conditioning session was ended at that point. The next session was held a week later. At the start of the session, the rat was handed to Albert to test his reaction to it. He tentatively reached for the rat, but quickly withdrew his hand after touching it. Since, by comparison, he showed no fear of some toy blocks that were handed to him, it seemed that a slight amount of fear conditioning to the rat had occurred during the previous week’s session. Albert was then subjected to further pairings of the rat with the noise, during which he became more and more fearful. Finally, at one point, when the rat was presented without the noise, Albert “began to crawl so rapidly that he was caught with difficulty before reaching the edge of the table” (Watson & Rayner, 1920, p. 5). Albert’s reaction was interpreted by Watson and Rayner as indicating that the rat had indeed come to elicit a conditioned fear response as a result of its association with the noise. This process can be diagrammed as follows: Rat (NS): Loud noise (US) ! Fear (UR) Rat (CS) ! Fear (CR) In subsequent sessions, during which Albert occasionally received additional conditioning trials, he reportedly showed not only a fear of the rat but also of objects that were in some way similar to the rat, such as a rabbit, a fur coat, a dog, and even a Santa Claus mask. In other words, Albert’s fear seemed to have generalized to objects that were similar to the original CS. His apparent fear of the rat, and his generalized fear of similar objects, persisted even following a 30-day break, although the intensity of his reactions was somewhat diminished. At that point, Albert left the hospital, so no further tests could be conducted. Watson and Rayner were also unable to carry out their original plan of using behavioral procedures to eliminate Albert’s newly acquired fear response.3 Although the Little Albert experiment is often depicted as a convincing demonstration of phobic conditioning in a young infant, this is actually 3 If all this seems terribly unethical, well, by today’s standards, it is. The lack of established ethical guidelines for psychological research in Watson’s era no doubt played a role. But it is also interesting to note that the Little Albert study hardly raised an eyebrow when it was published. In fact, Watson received far more criticism for his research with rats (from animal rights activists of that era) than he did for his research with Albert (Buckley, 1989). Although mistreatment of children was a concern at that time, people then were not as sensitive or aware as we are today about possible long-term problems resulting from adverse childhood experiences. 181 182 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications highly debatable (Harris, 1979; Powell, Digdon, Harris, & Smithson, 2014). For one thing, there was no control for the possibility of pseudoconditioning; that is, it is possible that during the conditioning trials, Albert was simply sensitized (upset) by the loud noise and it was this that resulted in his negative reaction to the relatively unfamiliar animals and objects he was being shown (as opposed to the specific pairing of the rat with the noise). It also took several pairings of the rat and the noise before the rat began to elicit a fear reaction. Albert’s fear reactions also tended to weaken considerably by the time of the next test session several days later. By contrast, real-life phobias usually require only one pairing of the CS with the aversive US to become established, and they often grow stronger over time. Watson and Rayner (1920) also noted that Albert was not afraid of anything as long as he was able to suck his thumb, and they had to repeatedly remove his thumb from his mouth during the sessions to enable a fear reaction to be elicited. This suggests that any fear conditioning that might have occurred was in fact relatively weak since it was easily countered by the comfort derived from thumb sucking. Further support for this conclusion is provided by a modern analysis of Watson’s (1923) film, Studies upon the Behavior of the Human Infant, which includes clips of the Little Albert experiment (Powell & Schmaltz, 2021). Although Albert does sometimes appear upset, many of these reactions are actually developmentally appropriate for an infant of that age. Children at about two years of age often go through a phase in which they react fearfully to unusual or unexpected events—such as a relatively unfamiliar animal or someone wearing a face mask—an innate process that Watson would have been unaware of, since the occurrence of such a phase was only later proposed by other researchers (e.g., Kagan, 1979; Scarr & Salapate, 1970; Valentine, 1930; refer to Figure 5.1). Thus, although Watson and Rayner (1920) speculated about the possibility of Albert growing up to be a neurotic individual with a strange fear of furry objects, there are strong reasons to believe that this did not occur. And even if he had acquired a conditioned fear response, additional factors are usually needed for such fears to develop into a true phobia, some of which are discussed in the next section.4 (For further information, including the identification of an individual who probably was the real Little Albert, review “On Searching for Little Albert: A Personal Account” in the And Furthermore box.) 4 It has been noted that the Little Albert study can also be interpreted as an example of operant conditioning (e.g., Goodwin, 2005). Because the loud noise occurred when Albert reached for the rat—meaning that the noise followed the reaching response and served to punish that response—the process can be described as an example of “positive punishment,” which is discussed in Chapter 6. 183 Prof. Benjamin Harris, University of New Hampshire 5.2 Practical Applications of Classical Conditioning Three film stills of Watson and Rayner with Little Albert during what was most likely the final test session. Although these images are almost always interpreted as illustrating Albert’s conditioned fear of rats and other furry animals and objects, this interpretation is highly questionable. In the first still, Albert is shown crawling, not away from the rat as Watson and Rayner (1920) reported, but over top of the rat that he is supposed to be afraid of (possibly because he was trying to crawl toward his mother, who may have been standing beside the camera). He then sits up and begins crying (as shown in the second still) only after Watson has pulled the rat out from under him and suddenly placed it near or on his hand, which may have startled him. The third still shows Albert’s reaction to a strange looking “Santa Claus” mask that Watson wears while crawling toward him. Although Albert looks concerned, he does not cry and is, if anything, acting quite rationally, given the circumstances. Many infants at the age find unusual objects and events, especially masks, to be extremely frightening. perspective, it seems to represent a process of over . 2. In the Little Albert experiment, the rat was originally a(n) stimulus. response, response. while his crying in response to the rat was a(n) 4. Differences between Albert’s fear conditioning and the conditioning of real-life phobias are that the latter often require (only one/more than one) conditioning trial and often grow (stronger/weaker) over time. 5. Albert’s fear response was (present/absent) whenever he was sucking his thumb, which suggests that the fear conditioning was actually relatively (strong/weak). 6. An additional problem with the Little Albert experiment is that many of the fear reactions Albert displayed were actually developmentally (appropriate/ inappropriate) for children at that age, who have an (innate/learned) tendency to react fearfully to unusual or unexpected events even in the absence of conditioning. stimulus, while the loud noise was a(n) 3. Albert’s startle response to the noise was a(n) Quick Quiz D 1. A phobia is an extreme, irrational fear reaction to an event. From a classical conditioning 184 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications And Furthermore On Searching for Little Albert: A Personal Account As noted earlier, Watson and Rayner (1920) reported that Little Albert’s mother removed him from the hospital before they could treat him for the phobia they believed they had instilled in him. As a result, people over the years have wondered whatever happened to Little Albert. Did he grow up with a fear of furry objects, as Watson and Rayner speculated he might? Unfortunately, various attempts to track him down were unsuccessful—that is, until recently. In some clever detective work, Beck, Levinson, and Irons (2009) used the 1920 U.S. census records to identify a woman, Arvilla Merritte, who (like Albert’s mother) resided in Johns Hopkins Hospital, where she worked as a wet nurse, during the time that the Watson and Rayner (1920) experiment was being conducted there. Hospital records indicated that Arvilla had given birth to a son, Douglas, whose age closely matched the reported age of Little Albert at the time the experiment began. Based on such congruencies, the authors concluded that Douglas was almost certainly Little Albert— and that the name given to him by Watson and Rayner, “Albert B,” was most likely a pseudonym. Not surprisingly, the reported discovery of Little Albert’s identity resulted in a great deal of fanfare: the APA Monitor, for example, proclaimed that “one of psychology’s greatest mysteries appears to have been solved” (DeAngelis, 2010, p. 10). But there was also a tinge of sadness and disappointment: Douglas had died from hydrocephalus at age six, and this in turn nullified any speculation about what had happened to Little Albert when he grew up. Not everyone, however, was convinced that the real Little Albert had been found (e.g., Harris, 2011; Powell, 2010, 2011; Reese, 2010). Although it seemed possible that Douglas Merritte was Little Albert, the evidence was perhaps not as strong as many people were assuming. These concerns grew when a follow-up article, three years later, reported some startling new evidence in the case (Fridlund, Beck, Goldie, & Irons, 2012). The article claimed that Douglas had suffered from hydrocephalus almost from birth and was severely ill during the time of the experiment; moreover, Watson almost certainly knew of the illness when selecting him for the experiment. In other words, Watson, in a severe breach of ethics, had knowingly experimented upon a neurologically impaired child and then deliberately hid this fact by describing the child as healthy! The evidence presented included an analysis of the Little Albert film that purportedly revealed previously unnoticed evidence of neurological impairment in Albert’s behavior. A subsequent examination of Douglas’s medical records, once thought to have been lost, confirmed that he had indeed been diagnosed with hydrocephalus soon after birth and had displayed numerous neurological symptoms throughout his stay in the hospital. Needless to say, this dramatic new version of the Little Albert story garnered considerable attention in the media, with Watson being roundly vilified for his unethical behavior (e.g., Coombs, 2013). Already having some doubts about the original claim that Douglas was Little Albert, I (Russ Powell) discussed my concerns about this case with a colleague, Nancy Digdon, who has a background in developmental psychology as well as an interest in history of 5.2 Practical Applications of Classical Conditioning 185 psychology. After viewing the Little Albert film herself and seeing no evidence of the neurological impairment that Fridlund et al. (2012) claimed to have seen, Nancy likewise became concerned that something was amiss. We decided to conduct a thorough analysis of the evidence for Douglas as Albert as well as search for an alternate candidate for Albert. Ben Harris, a noted Watson historian, and Christopher Smithson, a genealogical researcher, later joined the investigation. We began our search for Little Albert by focusing on another woman listed in the 1920 census record, Pearl Barger, who likewise resided in the hospital at the time of the Watson and Rayner experiment. Beck et al. (2009) had been unable to find any evidence that Pearl had had a baby with her during her stay in the hospital and therefore dismissed her as a possible candidate for Albert’s mother. In our own search, however, we came across a genealogical document posted on the Internet that indicated that Pearl Barger had married a Charles Martin in 1921 (less than a year after Pearl’s stay in the hospital) and that one of their children was named Albert! Further investigation revealed that their son’s full name was William Albert Martin (but almost everyone called him “Albert”) and that he had died in 2007. Most important, he was, like Douglas Merritte, about the correct age to have been Little Albert. Based on this information, we were eventually able to track down and interview Albert Martin’s niece, Dorothy Parthree. With her permission, we reviewed Albert Martin’s hospital records, which still existed in the medical archives of Johns Hopkins Hospital. The results of our investigation strongly suggest that Albert Martin was the real Little Albert (Powell et al., 2014; Digdon, Powell, & Harris, 2014). Key evidence includes the following: • mother and father were not yet married—which matches the name, “Albert B,” reported by Watson and Rayner (1920). It is worth noting that research articles from that era often used participants’ actual names or initials, there being no ethical requirement at that time to protect their identities. • child throughout much of his stay in the hospital, which matches the description of Little Albert given by Watson and Rayner. • the weight of 21 pounds reported by Watson and Rayner for Little Albert. This weight, which is heavy for a nine-month-old, is also consistent with Little Albert’s decidedly chubby appearance in the film. (Douglas Merritte, by contrast, was significantly underweight at that age due to his illness.) • months, 21 days of age when he was discharged from the hospital—which exactly matches the age reported by Watson and Rayner for when Little Albert was discharged. (Conversely, Douglas Merritte was discharged a week earlier.) Although the preponderance of evidence, at the time of this writing, strongly suggests that Albert Barger was the real Little Albert (Griggs, 2015), there remain some inconsistencies. For example, Watson (1924/1925) reported that Little Albert was “adopted by an out-of-town family” soon after leaving the hospital; Albert Barger, however, like Douglas Merritte, grew up with his mother. On the other hand, Albert was 186 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications Courtesy of Dorothy Parthree adopted in the sense that his father, Charles Martin, after marrying Albert’s mother, would have had to adopt him in order for his last name to be changed from Barger to Martin. If Watson heard rumors about this “adoption” and name change, he may have been misled into thinking that Albert, like many children of unwed mothers at that time, had been adopted by another family. So, if Albert Barger (Martin) was the real Little Albert, did he indeed grow up to have a fear of furry animals? In fact, his niece told us that he did have an aversion to dogs and other animals, which at first blush seems to confirm that the experiment had William Albert Martin (1919–2007) successfully (and unethically) instilled in him a lasting The person now believed to have phobia. At second blush, however, the evidence is been “Little Albert.” much less clear. His aversion to animals was more a dislike of animals than a fear of animals and it wasn’t particularly strong (his wife used to good-naturedly tease him about it). Moreover, as a child, he once had a dog that he very much loved, and he once told his niece that he acquired his dislike of dogs after the trauma of seeing the dog get run over in the street. Additionally, Albert Martin seems to have been a rather easygoing, if fastidious, individual who led a relatively satisfying life, which seems consistent with Watson and Rayner’s belief that the experiment would be unlikely to result in any lasting harm. Add to this the many weaknesses in the study, and it is impossible to say, one way or the other, whether the experiment had a lasting effect on him. But if there was an effect, it seems most likely to have been relatively minor. As far as we know, Albert Martin had no knowledge of spending the first year of his life in a hospital or of participating in Watson and Rayner’s (1920) experiment. His parents apparently kept it a secret from him. Interestingly, when his niece was asked in an interview what her uncle (an intellectually curious man who loved to read) would have thought about being Little Albert, she quickly replied, “Oh, he would have been thrilled” (Bartlett, 2014). Additional Factors in Phobic Conditioning Not all phobias are acquired through a direct process of classical conditioning. Indeed, many people with phobias are unable to recall any particular conditioning event before the development of their symptoms (Marks, 1969). Additionally, most people are surprisingly resilient when exposed to extremely frightening events and do not easily develop chronic fears such as seen in phobias and in posttraumatic stress disorder (PTSD). For example, the vast majority of people exposed to air raids during World War II endured them rather well, developing only short-term fears that quickly disappeared (Rachman, 1977). Researchers have therefore suggested several additional variables that, singly or in combination, may be involved in the development of a true phobia. These 5.2 Practical Applications of Classical Conditioning include observational learning, temperament, preparedness, incubation, US revaluation, and selective sensitization. Observational Learning Many phobias are acquired when observing fearful reactions in others. For example, in World War II, a major predictor of whether children developed a fear of air raids was whether their mothers were fearful. In addition, airmen who became phobic of combat often developed their symptoms after witnessing fear reactions in a crewmate (Rachman, 1977). This tendency to acquire conditioned fear reactions through observation may be inherited (Mineka, 1987). In such cases, a display of fear by another person can be conceptualized as an unconditioned stimulus that naturally elicits an unconditioned fear response in oneself: Display of fear by others (US) ! Fear in oneself (UR) A neutral stimulus that is associated with this fear display might become a conditioned stimulus for fear: Snake (NS): Display of fear by others (US) ! Fear in oneself (UR) Snake (CS) ! Fear in oneself (CR) The result is that a person who has had no direct confrontation with snakes may indirectly acquire a conditioned fear of snakes. (The other way in which observational learning may be involved in fear development is through higher-order conditioning, which is discussed in the section on observational learning in Chapter 11.) Temperament Temperament is one’s level of emotional reactivity, including how quickly and strongly one reacts to external stimulation. Temperament, which is largely inherited and can differ widely across individuals, can affect how easily a conditioned response is acquired, including a fear response. As noted in Chapter 4, Pavlov found that dogs that were shy and withdrawn conditioned more readily than dogs that were active and outgoing. Similarly, people with certain temperaments may be more susceptible than others to the development of conditioned fears (Clark, Watson, & Mineka, 1994). Even Watson, who downplayed the role of genetic influences in human behavior, acknowledged the possible influence of temperament. Watson and Rayner (1920) deliberately chose Albert as a subject for their experiment in the belief that his emotional stability would grant him a good deal of immunity against any harmful effects of their procedures. Preparedness The concept of preparedness refers to an innate tendency to learn certain behaviors or make certain types of associations more easily than others (Seligman, 1971). With respect to phobias, people in general may have an inherited tendency to make aversive associations to certain types of events, such as snakes and spiders as opposed to trees and flowers. This notion was initially proposed by Valentine (1930), who attempted to replicate Watson 187 188 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications Quick Quiz E and Rayner’s (1920) experiment with his one-year-old daughter. He did so by blowing a loud whistle whenever she touched certain objects. When the object she touched was a pair of opera glasses, she displayed no fear, even to the sound of the whistle. When the object was a caterpillar, however, some fear was elicited. Valentine also observed a two-year-old who became fearful of dogs “at slight provocation.” He concluded that humans have an innate tendency to fear certain objects or events, such as certain animals. More evidence for the role of preparedness in fear conditioning includes a study by Cook and Mineka (1989). They exposed laboratory-raised rhesus monkeys to videotapes edited to show another monkey reacting either fearfully or non-fearfully to either a fear-relevant stimulus (a toy snake or toy crocodile) or a fear-irrelevant stimulus (flowers or a toy rabbit). Only those monkeys who observed the model reacting fearfully to the fear-relevant stimulus acquired a conditioned fear reaction to that stimulus. Similarly, Soares and Öhman (1993) found that human subjects developed signs of conditioned anxiety when exposed to subliminal stimuli—pictures presented so briefly that subjects were consciously unaware of the content—that were paired with uncomfortable levels of electric shock. This conditioned reaction occurred when the pictures were of fear-relevant stimuli (snakes and spiders) as opposed to fear-irrelevant stimuli (flowers and mushrooms). This result supports the notion that humans, too, may be innately predisposed to learn to fear certain types of objects and events. As noted in our discussion of Little Albert, such fears may especially emerge in young children at around the age of two, but the innate tendency to readily learn certain fears, such as a fear of snakes or spiders, may remain with us throughout life. (The concept of preparedness is more fully discussed in Chapter 12.) Students often confuse the concepts of temperament and preparedness. In people, temperament refers to differences between people in how emotionally reactive they are, which in turn affects how easily they can develop any type of phobia. It is considered to be an important component of one’s personality. Preparedness (as it relates to phobias) refers to differences between phobias in how easily they can be acquired. Thus, temperament refers to how easily a certain person can acquire a phobia, while preparedness refers to how easily people in general can acquire a certain phobia. For example, the fact that Jason develops phobias more easily than does Samantha reflects the role of temperament; the fact that a phobia of snakes is more easily acquired than a phobia of toasters for people in general reflects the role of preparedness. 1. From a conditioning perspective, viewing a display of fear in others can stimulus that elicits a(n) be conceptualized as a(n) response of fear in oneself. The event the other person is stimulus that elicits reacting to might then become a(n) response of fear in oneself. a(n) 2. The term refers to an individual’s inherited level of emotionality and reactivity to stimulation. It (does/does not) seem to affect the extent to which responses can be classically conditioned. 5.2 Practical Applications of Classical Conditioning 3. The concept of p holds that, as a species, we are genetically programmed to acquire certain kinds of fears, such as fear of snakes and spiders, more readily than other kinds. 4. Travis was in a very serious accident yet had no qualms about driving the next day; Cam was in a minor accident and was petrified of driving for weeks afterward. These different outcomes may largely reflect (learned/inherited) differences in between the two individuals. t 5. The fact that people who live in cities are generally more fearful of encountering snakes than they are of being run over by a car, even though the latter is far more dangerous, may reflect (innate/learned) differences in p for acquiring certain kinds of fears. Incubation When a phobia develops through a direct process of classical conditioning, why does the conditioned fear response not extinguish with subsequent exposures to the CS? To some extent, extinction does not occur because the person tends to avoid the feared stimulus (the CS) so that repeated exposure to the CS in the absence of the US does not take place. Additionally, however, because of this tendency to move away from the feared stimulus, any exposures that do occur are likely to be very brief. According to Eysenck (1968), such brief exposures may result in a phenomenon known as “incubation.” Incubation refers to the strengthening of an already established conditioned fear response to an aversive CS (i.e., the feared object or event) as a result of repeated brief exposures to the CS by itself. For example, a child who is bitten by a dog and then repeatedly runs away each time a dog comes near him may find that his fear of dogs grows worse even though he is never again bitten. As a result, what may have started off as a moderate fear of dogs may evolve over time into a severe fear. In fact, this process might even result in a conditioned fear that is stronger than the unconditioned fear that was originally elicited when the child was bitten. It also contradicts the general rule that the presentation of the CS without the US will result in extinction (Sandin & Chorot, 2002). Note, too, that covert exposures to the feared stimulus—as in worrying about it—might also result in incubation (Wells & Papageorgiou, 1995). Incubation is, of course, one reason for the old adage that if you fall off a horse you should immediately get back on; if you wait, you might later become too fearful to get back on.5 US Revaluation As noted in Chapter 4, exposure to a US of a different intensity (i.e., a different value) than that used during conditioning can alter the strength of the response to a previously conditioned CS. This process 5 The term incubation is also used to refer to a closely related phenomenon, also of relevance to the get-back-on-the-horse adage, in which a conditioned fear response sometimes grows stronger during a rest period after conditioning without further exposures to either the US or the CS—a process that is believed to play a role in the delayed onset of PTSD that sometimes occurs following a traumatic event (e.g., Pickens, Golden, Adams-Deutch, Nair, & Shaham, 2009). 189 190 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications could play a major role in human phobias (Davey, 1992). Consider, for example, a skateboarder who experiences a minor injury as a result of a fall: Skateboarding: Minor injury ! Slight anxiety Skateboarding ! Slight anxiety Because the injury was relatively minor, skateboarding elicits only a slight amount of conditioned anxiety, most of which will likely extinguish as the skateboarder continues the activity. But imagine that this person later is in a car accident and suffers a severe injury: Car: Severe injury ! Strong anxiety In addition to now experiencing anxiety while being in a car— Car ! Strong anxiety —they might also display a strong degree of anxiety to skateboarding: Skateboarding ! Strong anxiety It is as though the skateboarder finally realizes just how painful an injury can be. And given that skateboarding is associated with being injured, it too now elicits strong feelings of anxiety. The process of US revaluation can also occur through observational learning. A student of one of the authors reported that she developed a phobia about snowboarding after first spraining her leg in a minor snowboarding accident —which resulted in only minor anxiety about snowboarding—and then witnessing someone else suffer a major snowboarding accident. In this circumstance, observational learning resulted in US inflation, which then led to the phobia. Note that this differs from pure observational learning to the extent that her earlier minor snowboarding accident set the stage for her later overreaction when witnessing someone else suffer a major snowboarding accident. Finally, US inflation can also occur through verbally transmitted information. Consider the following case described by Davey, de Jong, and Tallis (1993): M.!F. (male, aged 29 yr) worked as a bank employee. On one occasion the bank was robbed, and during the robbery M.!F. was threatened with a gun. He had not been particularly anxious at the time and returned to work the next day without complaining of any residual fear symptoms. However, 10 days after the robbery he was interviewed by the police, and during this interview he was told that he was very lucky to be alive because the bank robber was considered to be a dangerous man who had already killed several people. From this point on M.!F. did not return to work and developed severe PTSD symptoms. (p. 496) This example suggests that we have to be particularly careful about the sort of information that we convey to people who have experienced potentially traumatic events, because that information itself might induce a traumatic reaction. Indeed, research has shown that individuals who have 5.2 Practical Applications of Classical Conditioning 191 been exposed to a traumatic event and are then given a psychological debriefing (also known as critical incident stress debriefing)—which is a structured form of counseling designed to prevent the development of PTSD—are sometimes more likely to develop PTSD than those who do not receive such debriefings (e.g., Mayou & Ehlers, 2000; Sijbrandij, Olff, Reitsma, Carlier, & Gersons, 2006). It seems that a debriefing itself sometimes heightens the effect of the trauma, perhaps by giving victims the impression that the trauma was more severe than they would otherwise have thought. Although psychological debriefings are still used by some psychologists, more empirically based procedures have now been developed that are more respectful of a person’s individual coping style and are less likely to inadvertently do harm (Gist & Devilly, 2010). Selective Sensitization Yet another process that could influence the development of a phobia is selective sensitization, which is an increase in one’s reactivity to a potentially fearful stimulus following exposure to an unrelated stressful event. For example, people with agoraphobia (fear of being in a public place) often report that the onset of the disorder occurred during a period in which they were emotionally upset or suffered from some type of physical illness (Rachman, 1977). Similarly, an individual going through a stressful divorce might find that their usual minor anxiety about driving in heavy traffic suddenly develops into severe anxiety. The stressful circumstance surrounding the divorce affects their reactions not only to the divorce but to other potentially aversive events as well. Thus, during turbulent times in one’s life, minor fears and anxieties may become exacerbated into major fears and anxieties (Barlow, 1988). , in which the conditioned in a phenomenon known as fear response gradually grows (stronger/weaker). This runs counter to the general principle that presentation of the CS without the US usually results in . e 2. According to the concept of revaluation, phobic behavior might sometimes develop when the person encounters a (more/less) intense version of the (CS/US) than was used in the original conditioning. This process l or can also occur through o transmitted information. through v 3. The process of s s refers to an increase in one’s reactivity to a potentially fearful stimulus following exposure to a stressful event that is (related/unrelated) to the feared stimulus. Treating Phobias Perhaps more than any other disorder, phobias are highly susceptible to treatments based on behavioral principles of conditioning. In this section, we discuss the two classic forms of treatment: systematic desensitization and flooding. Quick Quiz F 1. Repeated brief exposures to a feared CS (in the absence of the US) may result 192 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications Systematic Desensitization Recall how Watson and Rayner had intended to treat the phobia they believed they had created in Albert but were unable to do so because his mother suddenly removed him from the hospital. A few years later, Mary Cover Jones (1924) did carry out such a treatment (under Watson’s supervision) with Peter, a two-year-old boy who had an extreme fear of rabbits. Jones’s treatment strategy consisted of first feeding Peter cookies while presenting a rabbit at a considerable distance. It was assumed that the positive emotional response elicited by the cookies would overcome the mild anxiety elicited by the distant rabbit. Over successive sessions, the rabbit was gradually brought closer to Peter as he continued to eat cookies. Within a few months, Peter was munching on cookies while holding the rabbit in his lap. As a result of this gradual conditioning procedure, Peter’s fear of the rabbit was eliminated. Although Jones’s treatment procedure, carried out in 1924, seemed to have effectively eliminated a phobia, it languished in obscurity until Joseph Wolpe (1958) essentially rediscovered it 30 years later. As a graduate student, Wolpe conducted research on fear conditioning in cats exposed to electric shocks. The cats displayed a strong fear of both the experimental chamber in which they had been shocked and the room containing the chamber. A major indication of this fear was the cats’ refusal to eat while in the room (which is an example of the conditioned suppression procedure that we discussed in Chapter 3). Wolpe then devised a treatment plan to eliminate the fear. He began by feeding the cats in a room that was quite dissimilar from the original “shock” room. Then, over a period of days, the cats were fed in rooms that were made progressively more similar to the shock room. Eventually they were able to eat in the room in which they had originally been shocked. The procedure effectively eliminated the conditioned fear in all 12 cats that Wolpe studied. The procedure that Wolpe (1958) used is called counterconditioning, in which a CS that elicits one type of response is associated with an event that elicits an incompatible response. In Wolpe’s study, the experimental room originally elicited a conditioned fear response because of its association with shock. Later, the room elicited a conditioned positive emotional response after it had become associated with food. Wolpe proposed that the underlying process in counterconditioning is reciprocal inhibition, in which the occurrence of one response can be inhibited by the simultaneous occurrence of an incompatible response. Thus, the positive emotional response elicited by food inhibited the cats’ anxiety because the two responses countered each other. As a result of his success, Wolpe (1958) began to ponder ways of applying this treatment procedure to human phobias. Although both he and Jones had successfully used the response of eating food to counter feelings of anxiety, Wolpe felt that this approach would be impractical for most treatment situations involving humans. He toyed with other types of responses that might counter anxiety, such as anger and assertiveness (i.e., the client was taught to act angry or assertive in situations that were normally associated with fear), but finally hit upon the use of deep muscle relaxation. Deep 5.2 Practical Applications of Classical Conditioning muscle relaxation is largely incompatible with the experience of anxiety (Jacobson, 1938), making it ideal from Wolpe’s perspective as a tool for counterconditioning. Wolpe (1958) also realized that real-life exposure to a phobic stimulus was impractical in many treatment scenarios. For example, it would be extremely difficult to expose a person with a fear of thunderstorms to a succession of storms that are made progressively more frightening. To solve this dilemma, Wolpe decided to have the patient simply visualize the feared stimulus. A series of visualized scenarios could then be constructed that would represent varying intensities of the feared event. For example, the person could imagine a storm some distance away that had only a mild amount of thunder and lightning, then a storm that was somewhat closer with a bit more thunder and lightning, and so on. One drawback to this procedure is that the counterconditioning occurs only to the visualized event, and it will then have to generalize to the real event. Nevertheless, if the visualization is fairly vivid, the amount of generalization to the real world should be considerable. Thus, Wolpe’s (1958) procedure, known as systematic desensitization, is a behavioral treatment for phobias that involves pairing relaxation with a succession of stimuli that elicit increasing levels of fear. The three parts of the procedure are as follows: 1. Training in relaxation. An abbreviated version of Jacobson’s (1938) deep muscle relaxation procedure is commonly employed for inducing relaxation, but other methods such as meditation or hypnosis have also been used. 2. Creation of a hierarchy of imaginary scenes that elicit progressively intense levels of fear. Experience has shown that about 10 to 15 scenes are usually sufficient, starting with a scene that elicits only a minor degree of fear (e.g., for a dog-phobic individual, it might be visualizing a friendly poodle tied to a tree at a distance of several yards) and finishing with a scene that elicits a tremendous amount of anxiety (e.g., visualizing standing beside a large dog that is barking). 3. Pairing of each item in the hierarchy with relaxation. Starting with the least fearful scene in the hierarchy, the person is asked to visualize the scene for about 10 to 30 seconds and then engage in a short period of relaxation. This process is repeated until the first scene no longer elicits anxiety, at which point the process is carried out using the next scene. By the time the top item in the hierarchy is reached, most of the person’s fear will have been eliminated, leaving only a residual amount of fear to what was once an intensely fearful scene. The fear response to this final scene is also eliminated, at which point it is quite likely that they will now feel significantly less anxious when confronted with the phobic stimulus in real life. Although Wolpe (1958) emphasized, mostly for convenience, the use of imaginary stimuli (the procedure then being referred to as imaginal desensitization), the treatment can also be carried out with real stimuli. This 193 194 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications version of desensitization is sometimes referred to as in vivo desensitization. Mary Cover Jones’s (1925) treatment of Peter’s rabbit phobia is an example of in vivo desensitization. As with imaginal desensitization, in vivo desensitization usually makes use of relaxation to counter the person’s fear response. For example, a dog-phobic client might move gradually closer to a real dog, pausing after each step and relaxing for several seconds. Additionally, the process might first be carried out with a very small dog and then gradually progress to a very large dog. In vivo desensitization has an obvious advantage in that one does not have to worry about whether the treatment effect will generalize to a real-life stimulus because one is already working with a reallife stimulus. A drawback, however, is that in severely phobic clients even minor amounts of the real stimulus (e.g., a small spider presented at a distance of several yards) might elicit tremendous amounts of anxiety. In such cases, it might be wise to first use imaginal desensitization to eliminate much of the fear, and then switch to in vivo desensitization to complete the process. More detailed information on systematic desensitization can be found in behavior modification texts such as Spiegler and Guevremont (2010). Considerable research has been carried out on systematic desensitization. It tends to be very effective with patients who have relatively few phobias that are highly specific in nature (e.g., a fear of spiders). By contrast, people diagnosed with social phobias, who tend to experience a generalized fear of many different social situations, do not respond as well to this form of treatment. Additionally, when using imaginal desensitization, the client must be able to clearly visualize the feared event and experience anxiety while doing so. Unfortunately, some individuals are unable to visualize clearly, or they feel no anxiety even with clear visualization. In these cases, in vivo desensitization is the better alternative. Since the advent of systematic desensitization, the basic method has been adapted to incorporate various technologies. One such method involves the use of virtual reality (VR) technology, which enables people to experience feared situations without real-world risk. VR desensitization provides more realism than imaginal desensitization and more safety and practicality than in vivo. For example, being able to confront a large dog or fly on a plane in a virtual scenario removes the risk that something negative will happen (such as a barking dog or a bumpy flight) that could inadvertently strengthen the phobia. VR is also ideal when a client has progressed beyond imaginal desensitization but isn’t quite ready for in vivo desensitization, or when in vivo desensitization is simply impractical or expensive (such as the cost of multiple airplane tickets to treat a fear of flying!). VR treatment procedures have become more popular as the technology has improved; studies of its efficacy indicate that for some phobias, it can be as effective as in vivo treatment (Wechsler, Kumpers, & Muhlberger, 2019). As noted, Wolpe (1958) assumed that systematic desensitization is a counterconditioning procedure that works through the process of reciprocal inhibition. However, some researchers (e.g., Eysenck, 1976) have claimed that systematic desensitization is simply a matter of extinction in which a 5.2 Practical Applications of Classical Conditioning 195 CS is repeatedly presented in the absence of the US. From this perspective, systematic desensitization for a dog-phobic individual works simply because it involves repeated presentations of dogs (or images of dogs) in the absence of anything bad happening, with the relaxation actually being irrelevant. In fact, relaxation is not always needed for the treatment to be effective; gradual exposure to the feared stimulus is by itself often sufficient. On the other hand, in support of the counterconditioning explanation, severe phobias often respond better to treatment when relaxation is included (Wolpe, 1995). The exact mechanism by which systematic desensitization produces its effects is, however, still unknown, and it may well be that both extinction and counterconditioning are often involved. response with an event that elicits an incompatible response is called . The process underlying this procedure is called c i , in which certain types r of responses that are (compatible/incompatible) with each other tend to each other. i 2. Mary Cover Jones used the response of to counter Peter’s d feelings of anxiety, while Wolpe, in his s . procedure, used 3. The three basic components of Wolpe’s procedure are: a. b. c. 4. A version of Wolpe’s procedure that uses real-life rather than imaginary stimuli desensitization. A major advantage of this is called procedure is that there is less worry about whether the treatment effect will to the real world. g 5. Wolpe’s procedure is very effective with people who have (few/many) phobias that are (general/specific) in nature. Thus, this procedure (does/does not) work well with people who have a social phobia. 6. One bit of evidence against the counterconditioning explanation for this type of treatment is that relaxation (is/is not) always necessary for the treatment to be effective. On the other hand, in keeping with the counterconditioning explanation, relaxation does seem to facilitate treatment when the phobia is (nonspecific/severe). Flooding Consider a rat that continues to avoid a goal box in which it was once shocked, even though no further shocks will ever be delivered. One straightforward way to eliminate this phobic behavior is to place the rat in the goal box and insert a barrier that prevents it from leaving. Forced to remain in the box, the rat will initially show considerable distress, but this will disappear as time passes and no shock is delivered. By simply preventing the avoidance response from occurring, we can quickly eliminate the rat’s fear. Quick Quiz G 1. The procedure of associating a stimulus that already elicits one type of 196 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications Study Tip: Counterconditioning is a process that can be applied to any situation that an individual finds aversive, including studying. Some students, for example, as a result of past frustrations, may find the act of studying distinctly unpleasant and even anxiety arousing. This contributes to their tendency to procrastinate, and when they do study, they find it hard to concentrate. Changing to a new study environment, discussed in Chapter 8, might help alleviate this problem. But another tactic that might help would be to associate studying with pleasant stimuli that will then counteract the aversive associations. Good students are often quite fussy about their study environment and arrange for it to be as pleasant as possible, with a comfortable chair, agreeable music, a favorite drink, and maybe even a treat to nibble on. If you presently find studying aversive, you might wish to consider doing likewise (with the treats being gradually phased out if you don’t want to get hooked on them). Practicing relaxation or meditation just prior to studying may also be helpful. As always, the trick is to experiment and find what works for you. The treatment procedure that makes use of this response-prevention principle is flooding therapy: a behavioral treatment that involves prolonged exposure to a feared stimulus, thereby providing maximal opportunity for the conditioned fear response to be extinguished (Spiegler & Guevremont, 2010). This method can be contrasted with systematic desensitization, in which exposure to the feared stimulus not only occurs gradually but also involves pairing the feared event with a response that will counteract the fear (such as relaxation). Flooding is more clearly based on the principle of extinction as opposed to counterconditioning. As with systematic desensitization, there are two basic types of flooding procedures. In imaginal flooding, the client is asked to visualize, as clearly as possible, a scenario involving the feared event. For example, an individual who is spider phobic might imagine waking up at night to find a large, hairy spider on the pillow beside them. A person with a fear of heights might imagine having to climb down a fire escape from a 10th-floor apartment. The greater the level of fear induced by the visualized scenario, the better. The client first visualizes the scenario in the therapist’s office and then practices visualizing it at home. Although the level of fear during visualization may initially increase, it should eventually begin to decrease. Once the fear response to one scenario has been extinguished, the fear response to other scenarios (e.g., having to remove a spider from the kitchen sink) can be similarly extinguished. After extinction has occurred within several scenarios, the client will likely experience considerably less fear when encountering the feared event in the real world. An alternative to imaginal flooding is in vivo flooding, which consists of prolonged exposure to the actual feared event. Consider, for example, a woman who is extremely fearful of balloons (perhaps because someone once burst a balloon in her face when she was a child). An in vivo flooding procedure might involve filling a room with balloons and then having the woman enter 5.2 Practical Applications of Classical Conditioning 197 the room, close the door, and remain inside for an hour or more. After a few sessions of this, her fear of balloons might well be eliminated. Of course, in vivo flooding is something that people have been intuitively aware of for centuries. The famous German poet and philosopher Goethe described how he cured himself of a fear of heights as a young man by climbing the tower of the local cathedral and standing on the ledge. He repeated this procedure until his fear was greatly alleviated (Lewes, 1965; review also “Was Sigmund Freud a Behavior Analyst?” in the And Furthermore box). As with in vivo desensitization, in vivo flooding is advantageous because it does not require the treatment effect to generalize from an imagined encounter to a real encounter. It is also not dependent on a person’s visualization ability. On the other hand, in vivo flooding can be highly aversive and cannot be used with some types of fears, such as house fires, that are impossible to replicate in a therapy setting. And Furthermore Was Sigmund Freud a Behavior Analyst? Students sometimes wonder how, if conditioning principles are so effective in treating certain disorders, other therapeutic systems that use decidedly different methods for treatment could have become so well established. One possibility is that these other systems sometimes make use of behavioral principles but have neglected to advertise the fact. For example, few people are aware that Sigmund Freud, the founder of psychoanalysis, very much appreciated the value of direct exposure to one’s fears. An example can be found in a description of Freud and his followers on a holiday outing in 1921 (Grosskurth, 1991). During an excursion in the mountains, they climbed a tower to a platform that was surrounded by an iron railing at hip level. Freud suggested that they all lean forward against the railing with their hands behind their backs, their feet well back, and imagine that there was nothing there to prevent them from falling. This was an exercise Freud had devised for overcoming the fear of heights, from which he had suffered as a young man. Jones [one of Freud’s most devoted followers] teased him that it didn’t seem very psychoanalytic. (p. 21) Despite Jones’s opinion, Freud (1919/1955) was so impressed with the effectiveness of this technique that he recommended it as an adjunct to psychoanalysis, arguing that a phobia first needs to be weakened through direct exposure before the final resolution of the fear can be achieved through psychoanalysis: One starts, therefore, by moderating the phobia [by direct exposure]; and it is only when that has been achieved at the physician’s demand that the associations and memories [of unconscious conflicts] come into the patient’s mind which enable the phobia to be resolved. (pp. 165–166) Of course, one might wonder how Freud could have determined that the resolution of the phobia was due to the retrieval of childhood memories rather than the cumulative effects of further exposure. (Consider also Thyer, 1999, for an example of how Carl Jung, another psychodynamic therapist, used an exposure-based procedure to treat a case of railroad phobia.) Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications www.CartoonStock.com 198 One concern with any type of flooding therapy is that the stress involved may result in medical complications. In addition, clients who have a history of other psychiatric disorders might experience an exacerbation of their fears. One must be particularly cautious about using flooding therapy to treat clients with PTSD. It is also important that the duration of each exposure, whether in vivo or imaginal, be sufficiently long (at least 30 to 45 minutes); otherwise, the fear may not be extinguished or, worse yet, become stronger. In this sense, flooding is a riskier procedure than systematic desensitization (Spiegler & Guevremont, 2010). 5.2 Practical Applications of Classical Conditioning 199 Hybrid Approaches to the Treatment of Phobias Systematic desensitization and flooding are the most basic behavioral approaches to the treatment of phobic behavior. Several variations of these approaches have been devised, which often combine aspects of each along with additional processes, such as observational learning. Such approaches are generally known as exposure-based treatments or exposure therapies and are now considered the treatment of choice for phobic disorders (Spiegler & Guevremont, 2010). For example, Öst (1989) described a method for rapidly eliminating specific phobias, such as a specific fear of spiders, in a single session. The major component of the treatment package was an in vivo exposure procedure in which clients were encouraged to approach the feared object as closely as possible, remain there until the anxiety faded away, and then approach the object even more closely. This process continued until the client had closely approached the object and had experienced at least a 50% reduction in their reported level of fear. Note that this exposure procedure is similar to systematic desensitization in that it is somewhat gradual, and similar to flooding in that the client is encouraged to endure a fairly intense level of anxiety each step of the way. Öst’s (1989) treatment package included several additional components. For example, throughout the procedure, most clients were accompanied by the therapist, who acted as a model to demonstrate how to interact with the feared object (such as how to use a jar to capture a spider). The therapist also helped the client physically contact the feared object—for example, by first touching the object while the client touched the model’s hand, then touching the object while the client also touched the object, and then gradually removing their hand while the patient continued touching the object. This procedure is sometimes called participant modeling (or contact desensitization) and has been shown to greatly facilitate treatment (Bandura, 1975; Bandura, Blanchard, & Ritter, 1969). 2. 3. 4. 5. 6. thereby providing maximal opportunity for the conditioned fear to . Two types of flooding therapy are flooding, in which one flooding, in which one visualizes the feared stimulus, and encounters a real example of the feared stimulus. For flooding therapy to be effective, the exposure period must be of relatively (long/short) duration. Unlike systematic desensitization, one (needs/does not need) to be cautious when employing this type of therapy with people who have a history of other types of psychiatric disorders. Modern therapies for phobias are often given the general name of e -b treatments. Öst’s single-session procedure combines the gradualness of s with the prolonged exposure time of fl . d m , This procedure also makes use of p in which the therapist demonstrates how to interact with the feared object. Quick Quiz H 1. In flooding therapy, the avoidance response is (blocked/facilitated), 200 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications Aversion Therapy for Problem Behaviors Some behavior problems stem from events being overly enticing rather than overly aversive. For example, nicotine and alcohol can be highly pleasurable, with the result that many people become addicted. Similarly, pedophiles have inappropriate feelings of sexual attraction to young children. Obviously, one way to counter these problem behaviors is to directly reduce the attractiveness of the relevant stimuli. Aversion therapy is a treatment procedure that reduces the attractiveness of a desired event by associating it with an aversive stimulus (Spiegler & Guevremont, 2010). An ancient version of this treatment was suggested by the Roman writer Pliny the Elder, who recommended treating overindulgence in wine by secretly slipping the putrid body of a large spider into the bottom of the wine drinker’s glass. The idea was that the feelings of revulsion elicited by a mouthful of spider would become associated with the wine, thereby significantly reducing the person’s desire for wine (Franks, 1963). More recent versions of this therapy are somewhat less primitive. For example, the taste of alcohol has sometimes been paired with painful electric shocks. An alternative version—which is similar to Pliny’s treatment in that it makes use of stimuli associated with ingestion—involves pairing the taste of alcohol with nausea. In this case, the client is first given an emetic, which is a drug that produces nausea. As the nausea develops, the client takes a mouthful of alcohol. This procedure is repeated several times, with the type of alcohol being varied across trials to ensure generalization. Research has shown that such nausea-based treatments are more effective than shock-based treatments, presumably because we have a biological tendency (i.e., a preparedness) to quickly associate nausea with substances that we ingest (Baker & Cannon, 1979; Masters, Burish, Hollon, & Rimm, 1987). This tendency, known as taste aversion conditioning, is discussed more fully in Chapter 12. Aversion therapy has also been used with smoking, with similar results. Early attempts to pair smoking and electric shock were relatively ineffective, possibly because physical pain is not a biologically relevant response to smoking. A more effective procedure has been to pair smoking with nicotineinduced nausea. This procedure, known as “rapid smoking,” involves having the client engage in a high rate of smoking, inhaling every 6 to 10 seconds (Danaher, 1977). Within a few minutes, extreme feelings of nausea will be elicited and the person is unable to continue. A single session is usually sufficient to produce at least temporary abstinence. This is especially the case with smokers who do not yet have a strong physical addiction to smoking and who smoke more for the pleasure of smoking—which the aversive conditioning counteracts—than for the avoidance of withdrawal symptoms (Zelman, Brandon, Jorenby, & Baker, 1992). Long-term abstinence is less certain but can be facilitated through the use of booster sessions (if the person is willing to go through it again) or relapse prevention training, in which the person learns to identify and cope with situations in which there is a high risk of resuming the problematic behavior (Marlatt & Gordon, 1985). Rapid smoking is, however, very stressful, and usually results in extreme increases in heart rate. Thus, this type of treatment must be employed cautiously, as it 5.2 Practical Applications of Classical Conditioning 201 could lead to medical difficulties (Lichtenstein & Glasgow, 1977). (In other words, do not try this at home!) Aversion therapy has also been used to treat sex offenders (Hall, Shondrick, & Hirschman, 1993). In the case of pedophiles, photographic images of unclothed children may be paired with drug-induced nausea or a powerfully unpleasant scent, such as ammonia. As part of a comprehensive treatment package, such procedures help reduce the risk that the individual will reoffend following release from prison.6 Aversion therapy is sometimes carried out with the use of imaginal stimuli rather than real stimuli. This version of the treatment is usually called covert sensitization. For example, a person addicted to smoking might imagine experiencing extreme illness and vomiting each time they try to smoke. Alternatively, they might visualize being forced to smoke cigarettes that have been smeared with feces. As with imaginal desensitization, the effectiveness of this procedure is dependent on the client’s ability to visualize images clearly and to experience strong feelings of revulsion in response to these images. The treatment effect also has to generalize from the visualized event to the real event, which, as in imaginal treatments for phobias, is likely to result in some loss of effectiveness. Thus, covert sensitization will likely be somewhat less effective than aversion therapy, which utilizes exposure to the actual stimulus. As with the treatment of phobias, however, virtual reality approaches to covert sensitization can help alleviate such problems and have shown considerable promise in the treatment of addictive disorders (Segawa et al., 2020). 2. 3. 4. 5. therapy, one attempts to reduce the attractiveness of a highly desired or addictive event by associating that event with an unpleasant stimulus. A standard treatment for alcoholism is to associate the taste of alcohol with that have been induced by consumption of feelings of n . an e A highly effective procedure for reducing cigarette consumption, at least temporarily, s . is r In general, aversion therapy is (more/less) effective when the unpleasant response that is elicited is biologically relevant to the problematic behavior. Aversion therapy is sometimes carried out using stimuli rather than real stimuli. This type of treatment procedure is known as sensitization. Medical Applications of Classical Conditioning There is a growing body of evidence indicating that processes of classical conditioning have significant medical implications. For example, Russell et al. (1984) were able to condition guinea pigs to become allergic to certain odors 6 Although aversion therapy for pedophiles does reduce the likelihood that they will reoffend, be aware that these treatments have not been shown to provide a lasting “cure” for most offenders (Kirsch & Becker, 2006). Quick Quiz I 1. In 202 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications by pairing those odors with an allergy-inducing protein. People who have allergies may experience a similar process, such that their allergic reaction is elicited not only by the substance that originally caused the allergy but also by stimuli associated with it. Thus, for a person who is allergic to pollen, even the mere sight of flowers might elicit an allergic reaction. Flowers (NS): Pollen (US) ! Allergic reaction (UR) Flowers (CS): ! Allergic reaction (CR) Other studies have shown that various aspects of the immune system can be classically conditioned. For example, Ader and Cohen (1975) exposed rats to an immunosuppressive drug paired with saccharin-flavored water. These rats were then given an injection of foreign cells, followed by a drink of either saccharin-flavored water or plain water. The rats that drank the saccharinflavored water produced fewer antibodies in reaction to the foreign cells than did the rats that drank the plain water. The flavored water had apparently become a CS for immunosuppression. In a real-world extension of this study, Bovbjerg et al. (1990) found that women who received chemotherapy in a hospital setting displayed evidence of immunosuppression when they later returned to the hospital. The hospital environment had become associated with the immunosuppressive effect of the chemotherapy and was now a CS for a conditioned immunosuppressive response. Thus: Hospital (NS): Chemotherapy (US) ! Immunosuppression (UR) Hospital: (CS) ! Immunosuppression (CR) Other studies have shown that classical conditioning can be used to strengthen immune system functioning. For example, one team of researchers gave human subjects a taste of sherbet followed by shots of epinephrine (BuskeKirschbaum, Kirschbaum, Stierle, Jabaij, & Hellhammer, 1994). Epinephrine tends to increase the activity of natural killer cells, which are an important component of the body’s immune system. After pairing the sweet sherbet with the epinephrine, the sweet sherbet itself elicited an increase in natural killer cell activity. Hence: Sweet sherbet (NS): Epinephrine (US) ! Increased killer cell activity (UR) Sweet sherbet (CS) ! Increased killer cell activity (CR) (Refer also to Solvason, Ghanta, & Hiramoto, 1988.) The medical implications of such findings are significant. Many patients would benefit considerably from enhanced immune functioning during the course of their illness. Other patients, however—namely those with autoimmune diseases, such as arthritis, in which the immune system seems to be overactive—would benefit from a procedure that could weaken their immune system. (Refer to Exton et al., 2000, for a review of research into this issue; also Ader, 2003.) 5.2 Practical Applications of Classical Conditioning 203 Classical conditioning also has important implications for our understanding of the placebo effect (Siegel, 2002). In drug research, a placebo is an inert substance that appears to be a drug but in reality has no pharmacological value. In double-blind control studies, placebos are given to a control group to assess the effects of “expectancy” upon the patient’s symptoms, such effects being known as placebo effects. Only when the drug effect is stronger than the placebo effect is the drug considered effective. In classical conditioning terms, the placebo effect can be seen as the result of pairing the appearance of the drug (originally an NS) with the active ingredients of the drug (the US). Thus, conditioning a placebo effect for aspirin, in which the active ingredient is acetylsalicylic acid, would involve the following: White pill (NS): Acetylsalicylic acid (US) → Headache removal (UR) White pill (CS) → Headache removal (CR) The possibility that this type of process underlies at least some instances of the placebo effect is supported by the fact that such effects are much more likely to occur following a period of treatment with the active drug (e.g., Kantor, Sunshine, Laska, Meisner, & Hopper, 1966). Additionally, repeated administration of a placebo by itself tends to reduce its effectiveness, which suggests that a process of extinction is taking place (Lasagna, Mosteller, von Felsinger, & Beecher, 1954). (Note, however, that this would work quite differently for people who have developed a strong tolerance to a drug. In!those cases, as described earlier in this chapter, the fake drug would probably act as a CS that elicits a compensatory CR in the form of a craving or withdrawal.) If conditioning processes do underlie placebo effects, research into this process might allow us to better control such effects. Placebos could then be used, for example, to reduce the frequency with which a patient has to take the real drug, thereby reducing some of the side effects associated with the drug. Additionally, we may be able to devise ways in which the placebo effect can be combined with the real drug to produce an enhanced form of treatment (refer to Siegel, 2002). immunosuppression. Label each component using the appropriate abbreviations. Saccharine flavor (NS): (US) → (UR) (CS) → (CR) 2. Supporting the possibility that placebo effects are classically conditioned responses, such effects are more likely to occur (following/preceding) a period of treatment with the real drug. As well, repeated presentations of the placebo by itself tend to (reduce/increase) its effectiveness, which suggests that is taking place. e Quick Quiz J 1. Diagram the classical conditioning process in Ader and Cohen’s (1975) study of 204 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications In review To create a summary for this chapter, include a paragraph that briefly defines stimulus-substitution theory, preparatory-response theory, and the Rescorla– Wagner theory. A second paragraph should describe how drug tolerance is learned and how withdrawal and overdose can be understood using the compensatory-response model. Use two paragraphs to describe how fear and phobias may result from learning and how they can be treated or reduced with conditioning procedures. Finally, you could have one paragraph that describes aversion therapy and various applications of conditioning in medicine. You can also test your understanding by creating plans to treat your own fears or problematic behaviors, using methods from this chapter. You should be able to diagram your plan and indicate which theories are most applicable. Further Exploration Alcohol and drug use are problems for a significant number of people. For information about how to get help, you can start with these government sites: In the United States: www.usa.gov/mental-health-substance-abuse. In Canada: www.canada.ca/en/health-canada/services/substance-use/get-help /get-help-problematic-substance-use.html. Video game designers are very interested in the application of VR to learning, including the learning of fears and phobias, and how technology interacts with our emotions. This video highlights several examples of game designs that aim to help people deal with fear while playing video games as well as how games are being created to specifically treat phobias and PTSD: www.youtube.com/watch?v5nduvjlw7du0. Study Questions 1. Describe Pavlov’s stimulus-substitution theory. What is the preparatoryresponse theory of conditioning, and what advantage does it have over stimulus-substitution theory? 2. Describe the compensatory-response model of conditioning. How does the compensatory-response model account for drug overdoses that occur when a user seems to have injected only a normal amount of the drug? 3. Describe the Rescorla–Wagner theory. Describe (or diagram) the Rescorla–Wagner theory account of blocking. Describe (or diagram) how the Rescorla–Wagner theory accounts for the overexpectation effect. 4. Briefly describe the Watson and Rayner experiment with Little Albert and the results obtained. What are some of the weaknesses in this experiment as a demonstration of classical conditioning of a phobia? Concept Review 5. Assuming that a look of fear in others can act as a US, diagram an example of observational learning in a child’s acquisition of a phobia to mice. Be sure to include the appropriate abbreviations (NS, US, etc.). 6. Describe how temperament and preparedness can affect the acquisition of a phobia, and give an example of each that helps clearly differentiate between the two. 7. Describe how incubation, selective sensitization, and US revaluation can affect the acquisition of a phobia. Give an example of each that clearly differentiates it from the other two concepts. 8. What is counterconditioning and reciprocal inhibition? Describe how counterconditioning might be used to try to reduce your aversion to your roommate’s bad taste in music. 9. Describe the use of imaginal versions of systematic desensitization versus flooding to treat a student’s exam anxiety. How do they differ in terms of the underlying process by which each type of treatment is presumed to produce its effect? 10. Define aversion therapy and covert sensitization. Why is shock generally less effective than nausea in producing an aversion to cigarettes or alcohol? 11. Diagram an example of a classical conditioning procedure that alters immune system functioning. In what way might classical conditioning be involved in the creation of a placebo effect? Concept Review aversion therapy. A form of behavior therapy that attempts to reduce the attractiveness of a desired event by associating it with an aversive stimulus. compensatory-response model. A model of conditioning in which a CS that has been repeatedly associated with the primary response (a-process) to a US will eventually come to elicit a compensatory response (b-process). counterconditioning. The procedure whereby a CS that elicits one type of response is associated with an event that elicits an incompatible response. flooding therapy. A behavioral treatment for phobias that involves prolonged exposure to a feared stimulus, thereby providing maximal opportunity for the conditioned fear response to be extinguished. incubation. The strengthening of an already established conditioned fear response to an aversive CS (i.e., the feared object or event) as a result of repeated brief exposures to the CS by itself. overexpectation effect. The decrease in the strength of a CR that occurs when two separately conditioned CSs are combined into a compound stimulus for further pairings with the US. preparatory-response theory. A theory of classical conditioning that proposes that the purpose of the CR is to prepare the organism for the presentation of the US. 205 206 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications preparedness. An innate predisposition to learn certain kinds of associations more easily than others (e.g., to associate snakes with an aversive event). reciprocal inhibition. The process whereby the occurrence of a response is inhibited by the simultaneous occurrence of an incompatible response. Rescorla–Wagner theory. A theory of classical conditioning that proposes that a given US can support only so much conditioning and that this amount of conditioning must be distributed among the various CSs. selective sensitization. An increase in one’s reactivity to a potentially fearful stimulus following exposure to an unrelated stressful event. stimulus-substitution theory. A theory of classical conditioning that proposes that the CS acts as a substitute for the US. systematic desensitization. A behavioral treatment for phobias that involves pairing relaxation with a succession of stimuli that elicit increasing levels of fear. temperament. An individual’s level of emotional reactivity, including how quickly or strongly one reacts to external stimulation. Chapter Test 8. The three steps in systematic desensitization are training in , creation of a of feared situations, and pairing . 21. In the Little Albert study, the loud noise was the (CS/US), while the white rat was the (CS/US). Little Albert’s fear of other furry objects illustrates the process of stimulus . 3. Lothar’s job has recently become quite stressful. Interestingly, he also finds that he is becoming more fearful of driving on icy roads. This is best described as an example of . 12. One weakness in the Little Albert study is that they did not control for the possibility of conditioning. Also, unlike most real-life phobias, Albert’s fear conditioning seemed to require (several/ one) trial(s) to become established and grew (weaker/stronger) over time. 25. Tara’s original slight fear of spiders turns into a major phobia when she witnesses a friend become hospitalized after being bitten by a spider. This is an example of . 7. The procedure of pairing a feared dog with an appetitive stimulus such as candy is an example of . The effectiveness of this procedure for reducing fears is due to the underlying process of . 20. When Uncle Bob and Aunt Shirley were separated, they each gave Little Lucas great Christmas presents, with the result that he developed positive feelings for both of them. They then resolved their difficulties and moved back together. They now give Little Lucas one great present from the two of them. The Rescorla–Wagner theory predicts that Little Lucas’s positive feelings for each will become (stronger/weaker). This is known as the effect. Chapter Test 13. In keeping with theory, Pavlov’s dogs would sometimes treat a light that had been paired with food as though it were itself food. 9. Desensitization and flooding procedures that utilize covert images of the feared stimulus are known as procedures, whereas procedures that involve exposure to the real stimulus are known as procedures. 2. While playing with a spider, Suyen was frightened by the sound of a firecracker. As a result, she acquired a lasting fear of spiders, but not of firecrackers. This is an example of the concept of . 17. According to the Rescorla–Wagner theory, overshadowing occurs because the stimulus picks up most of the associative value. 26. Fatalities seemingly due to drug overdose may sometimes be the result of taking the drug in a setting that is (associated/not associated) with drug use, thereby resulting in a (weaker/stronger) compensatory response and a (higher/lower) level of drug tolerance. 10. In therapy, one attempts to (decrease/increase) the attractiveness of a highly desired or addictive event by pairing it with an (appetitive/aversive) stimulus. An imagery-based form of this therapy is called . 6. Traditional advice has it that if you fall off a horse, you should immediately get back on and keep riding until your fear has disappeared. This approach is similar to the therapeutic technique known as . Furthermore, immediately getting back on the horse allows no opportunity for the occurrence of (prolonged/brief/ repeated brief) exposures to the feared stimulus, which could result in of the conditioned fear response. 24. Evidence for the role of conditioning in placebo effects includes the fact that such effects are more likely to occur (following/preceding) a period of treatment with (a fake/the real) drug. Also, repeated administration of a placebo reduces its effectiveness, which suggests that a process of is taking place. 14. I am likely to become most intoxicated if I drink alcohol in the presence of cues (associated with/not associated with) alcohol. 18. According to the Rescorla–Wagner theory, occurs because the (CS/NS/US) in the compound stimulus during conditioning has already picked up most of the associative value. 4. Bo was never afraid of bees until he saw his best friend, Emmet, react with a look of horror to the sight of a bee. Bo now becomes quite anxious each time he sees a bee. This is best described as an example of learning. 15. A cat salivates to the sound of your alarm clock in anticipation of a breakfast feeding. It also freezes at the sight of another cat in anticipation of an attack. These examples are best accounted for by the theory of conditioning. 207 208 Chapter 5 Classical Conditioning: Underlying Processes and Practical Applications 23. Tika’s slight fear of snakes turns into a major phobia when she develops a serious illness. This is an example of the process of . 1. The ease with which an individual can acquire a conditioned fear response can be influenced by that person’s tendency to be emotionally reactive, which is known as the person’s . This reactivity is largely (inherited/learned). 11. Fran experiences an allergic reaction even when she hears people talk about dogs. In the terminology of classical conditioning, the talk about dogs appears to be a (use the abbreviation) while the allergic reaction is a . 19. According to the effect, if two fully conditioned stimuli are combined into a compound stimulus that is then subjected to further pairings with the US, the associative value of each member of the compound will (increase/decrease). 5. In conditioning of a compensatory heart rate response to shock, the actual US for the compensatory response would be the (shock/anticipatory decrease in heart rate/shock-induced increase in heart rate). 16. Research on classical conditioning processes in drug addiction suggests that the withdrawal symptoms evoked by the sight of a desired drug are actually reactions to the drug that have come to be elicited by environmental cues associated with the drug. 27. One problem with the Little Albert experiment is that it did not control for the fact that children of that age have a(n) (learned/innate) tendency to react fearfully to relatively unfamiliar or unexpected objects and events. 22. Tran’s slight fear of rats turns into a major phobia when he is told by his parents that rats are much more dangerous than he previously suspected. This is an example of . Answers to Chapter Test 1. temperament; inherited 2. preparedness 3. selective sensitization 4. observational 5. shock-induced increase in heart rate 6. flooding; repeated brief; incubation 7. counterconditioning; reciprocal inhibition 8. relaxation; hierarchy; each item in the hierarchy with relaxation 9. imaginal; in vivo 10. aversion; decrease; aversive; covert sensitization 11. CS; CR 12. pseudo-; several; weaker 13. stimulus-substitution 14. not associated with 15. preparatory-response 16. compensatory (or opponent or b-process) 17. more salient (stronger) 18. blocking; CS 19. overexpectation; decrease 20. weaker; overexpectation 21. US; CS; generalization 22. US revaluation 23. selective sensitization 24. following; the real; extinction 25. US revaluation (in this case combined with observational learning) 26. not associated; weaker; lower 27. innate Chapter 6 Operant Conditioning: Introduction “Hurry up,” he growled as she carefully searched the selection of magazines. “Oh, don’t be so grumpy,” she said sweetly, hooking her arm into his. “Just pick one, damn it!” She quickly picked out a magazine, then gave him a hug as they walked to the checkout counter. (This is based on a real incident observed in a store.) Chapter Outline 6.1 Historical Background Thorndike’s Law of Effect Skinner’s Selection by Consequences 6.2 Operant Conditioning Operant Behavior Operant Consequences: Reinforcers and Punishers Operant Antecedents: Discriminative Stimuli 6.3 Four Types of Contingencies Positive Reinforcement Negative Reinforcement Positive Punishment Negative Punishment 6.4 Positive Reinforcement: Further Distinctions Immediate Versus Delayed Reinforcement Primary and Secondary Reinforcers Intrinsic and Extrinsic Reinforcement Natural and Contrived Reinforcers Shaping Learning Objectives 6-1 Distinguish between operant (instrumental) conditioning and classical conditioning. 6-2 Describe Thorndike’s law of effect and his classic experiment that led to its creation. 6-3 Describe Skinner’s approach to operant conditioning, including his emphasis on observable behavior. 6-4 Define operant behavior, reinforcers, and punishers. 209 210 Chapter 6 Operant Conditioning: Introduction 6-5 Explain the role of discriminative stimuli within the three-term contingency, including contingencies of extinction. 6-6 Describe and distinguish among the four types of operant contingencies. 6-7 Describe the different types of reinforcement, including primary/ secondary, immediate/delayed, intrinsic/extrinsic, and natural/contrived reinforcement. 6-8 Describe shaping procedures, including situations that require shaping. In the last few chapters, we focused on elicited behavior and the type of learning known as classical conditioning. Elicited behavior is controlled by the stimuli that precede it. Recall how in Pavlov’s classic experiment food elicited salivation and how, after a tone had been paired with food, it too elicited salivation: Tone: Food → Salivation Tone → Salivation Quick Quiz A Note how the target response in this type of learning always occurs at the end of the sequence. The preceding stimulus, by itself, is sufficient to elicit the response. In this sense, the process is very reflexive: present the stimulus and the response automatically follows. But is everything we do this reflexive? Does the sight of this text, for example, automatically elicit the response of reading? Obviously it does not (though students who tend to procrastinate might sometimes wish that it did). Rather, if you had to explain why you are reading this text, you are likely to say you are reading it in order to achieve something—such as an understanding of the subject matter or a high grade in a course. Reading the text is oriented toward some goal, a consequence, and this consequence is the reason for the behavior. Indeed, most behaviors that concern us each day are motivated by a consequence. For example, we go to a restaurant for a meal, we turn on a radio to hear music, and we ask someone out on a date hoping they will accept. When we fail to achieve the desired outcome, we are unlikely to continue the behavior. How long would you persist in turning on the radio if it never produced any music? Behaviors that are influenced by their consequences are called operant behaviors and the effects of those consequences upon behavior are called operant conditioning. They are called operant conditioning because the response operates on the environment to produce a consequence. This type of learning is also called instrumental conditioning because the response is instrumental in producing the consequence. 1. Operant behaviors are influenced by their . 2. Elicited behavior is a function of what (precedes/follows) it; operant behavior is a function of what (precedes/follows) it. 3. Another name for operant conditioning is conditioning. 6.1 6.1 Historical Background Historical Background Photo Researchers/Science History Images/Alamy Stock Photo Although people have used operant conditioning for thousands of years (e.g., in raising children, training animals), this kind of learning was not subjected to scientific analysis until the 1890s, when Edwin L. Thorndike investigated the learning ability of animals. Thorndike’s Law of Effect As a graduate student, Thorndike was interested in animal intelligence. There was considerable speculation at that time that animals were capable of higher forms of reasoning. Particularly impressive were stories about lost dogs and cats finding their way home over long distances. As Thorndike (1898) noted, however, “Dogs get lost hundreds of times and no one ever notices it or sends an account of it to a scientific magazine, but let one find his way from Edwin L. Thorndike Brooklyn to Yonkers and the fact immediately (1874–1949) becomes a circulating anecdote” (p. 4; refer also to Thorndike, 1911). Thorndike was not suggesting that animals could not in some ways be intelligent, but rather that we should not accept anecdotes as fact, nor should we assume that animals behaving in a particular way are doing so for intelligent reasons. It was not only the lay public that caused Thorndike to argue for caution in interpreting animal behavior. Some of his contemporary researchers were also guilty of a noncritical analysis of animal intelligence. In particular, George John Romanes (1888/1989) argued that animals engage in thought processes, such as decision making, that are analogous to human thought processes, and he presented various anecdotes to support his case. Thorndike was skeptical, however, and believed that the intellectual ability of animals could be properly assessed only through systematic investigation. Of the many experiments Thorndike (1898) conducted with animals, the most famous involved cats. In a typical experiment, a hungry cat was enclosed in a puzzle box, and a dish of food was placed outside. To reach the food, the cat had to learn how to escape from the box, such as by stepping on a treadle that opened a gate. The first time the cat was placed in the puzzle box, several minutes passed before it accidentally stepped on the treadle and opened the gate. Over repeated trials, it learned to escape the box more quickly. There was, however, no sudden improvement in performance as would be expected if the cat had experienced a “flash of insight” about how to solve the problem. Rather, it seemed as though the response that worked (stepping on the!treadle) was gradually strengthened, while responses that did not work 211 Chapter 6 Operant Conditioning: Introduction DILBERT © 1993 Scott Adams. Used by permission of UNIVERSAL UCLICK. All rights reserved 212 A convincing example of animal intelligence. Figure 6.1 Thorndike’s puzzle box. In a typical experiment, a hungry cat was enclosed in a puzzle box and a dish of food was placed outside the box. To reach the food, the cat had to learn how to escape from the box by stepping on a treadle that opened a gate. The graph illustrates the general decrease across trials in the amount of time it took the cat to escape. (Source: Nairne, 2000.) 600 Time required to escape (sec) 500 400 300 200 100 0 10 20 30 40 50 60 70 Trials (e.g., clawing at the gate, chewing on the cage) were gradually weakened (refer to Figure 6.1). Thorndike suspected that a similar process governed all learning, and on this basis he formulated his famous law of effect.1 1 Although Thorndike’s research led to a general tendency to reject anecdotal evidence for “insightful” learning in animals, some researchers believe he may have overstated the case. They claim that there is evidence available for rapid learning, depending on the task and the species examined (refer to Wasserman & Zentall, 2006, for a comprehensive review). 6.1 Historical Background According to Thorndike’s law of effect, behaviors leading to a satisfying state of affairs are strengthened, or “stamped in,” while behaviors leading to an unsatisfying or annoying state of affairs are weakened, or “stamped out.” Thus, the extent to which the consequences of a behavior are satisfying or annoying determine whether the behavior will be repeated. Thorndike’s law of effect is a hallmark in the history of psychology. However, it was another young scientist by the name of Burrhus Frederic Skinner who fully realized the implications of this principle for understanding and changing behavior. Skinner’s Selection by Consequences Skinner came upon the study of operant conditioning by a somewhat different route. As a graduate student in the late 1920s, he was well aware of Thorndike’s law of effect. However, like many psychologists of the time, he believed that behavior could best be analyzed as though it were a reflex. He also realized, like Pavlov, that a scientific analysis of behavior required finding a procedure that yielded regular patterns of behavior. Without such regularity, which could be achieved only in a well-controlled environment, it would be difficult to discover the underlying principles of behavior. In this context, Skinner set out to devise his own procedure for the study of behavior, eventually producing one of the best-known apparatuses in experimental psychology: the operant conditioning chamber, or “Skinner box.” In a standard Skinner box for rats, the rat is able to earn food pellets by pressing a response lever or bar (refer to Figure 6.2). Skinner’s procedure is known as the “free operant” procedure because the rat freely responds with a particular behavior (like pressing a lever) for food, and it may do so at any rate. The experimenter controls the contingencies within the operant chamber, but the animal is not forced to respond at a particular time. This contrasts with other procedures for studying animal learning, such as maze learning, in which the experimenter initiates each trial by placing the rat in the start box.2 Skinner demonstrated that the rate at which the rats pressed the lever to earn food was controlled by the conditions that he established in his experiments. Later, Skinner invented a variant of the operant chamber for pigeons, in which the pigeon pecks an illuminated plastic disc called a response key (named after the telegraph key) to earn a few seconds of access to food (refer to Figure 6.3). Many of the principles of operant conditioning were discovered with the use of these key-pecking and lever-pressing procedures. With the evolution of the Skinner box, Skinner’s beliefs about the nature of behavior also changed. He abandoned the notion that all behavior could be analyzed in terms of reflexes and, along with other learning theorists, came to 2 Although the terms operant conditioning and instrumental conditioning are often used interchangeably, the latter term is sometimes reserved for procedures that involve distinct learning trials, such as maze learning experiments, as opposed to Skinner’s free operant procedure. 213 214 Chapter 6 Operant Conditioning: Introduction Operant conditioning chamber for rats. When the rat presses the lever (or bar), a food pellet drops into the food tray. Aversive stimuli can be presented by delivering an electric shock through the floor grids. (Source: Lieberman, 2000.) Figure 6.2 Houselight Lever Food pellets Food tray Operant conditioning chamber for pigeons. When the pigeon pecks the response key (a translucent plastic disc that can be illuminated with different colored lights), grain is presented in the food cup for a period of a few seconds. (Source: Domjan, 2000.) Figure 6.3 Pecking key Food cup believe that behaviors can be conveniently divided into two categories. One category consists of involuntary, reflexive-type behaviors, which, as Pavlov demonstrated, can often be classically conditioned to occur in new situations. Skinner referred to such behavior as respondent behavior. The other category, which Skinner called operant behavior, consists of behaviors that seem more 6.2 Operant Conditioning 215 voluntary in nature and are controlled by their consequences rather than by the stimuli that precede them. It was this type of behavior that Thorndike had studied in his puzzle box experiments and upon which he had based his law of effect. It was this type of behavior that most interested Skinner as well. He spent the rest of his life investigating the basic principles of operant conditioning and applying those principles to important aspects of human behavior (Skinner, 1938, 1967; refer also to Bjork, 1993). 3. 4. 5. 6. 7. of , which states that state of affairs are behaviors that lead to a(n) state of strengthened, while behaviors that lead to a(n) affairs are weakened. According to Thorndike, behaviors that worked were st , while behaviors that did not work were i o . st The Skinner box evolved out of Skinner’s quest for a procedure that would, among other things, yield (regular/irregular) patterns of behavior. In the original version of the Skinner box, rats earn food by al ; in another version, p pigeons earn a few seconds of access to food by p at an illuminated plastic disc known as a r . k Skinner’s procedures are also known as fr procedures in that the animal controls the rate at o which it earns food. Skinner originally thought all behavior could be explained in terms of , but he eventually decided that this type of behavior could be distinguished from another, seemingly more voluntary type of behavior behavior. known as 6.2 Operant Conditioning Operant conditioning is a type of learning in which the future frequency (or probability) of a behavior is affected by its consequences. Note that this is essentially a restatement of Thorndike’s law of effect. Skinner, however, was dissatisfied with Thorndike’s mentalistic description of consequences as being either satisfying or annoying. Satisfaction and annoyance are internal states inferred from the animal’s behavior. Skinner avoided any speculation about what the animal (or person) might be thinking or feeling and reworded the law of effect to emphasize the effect of the consequence on the future probability of the behavior. Note that Skinner’s principle of operant conditioning bears a striking resemblance to Darwin’s evolutionary principle of natural selection. According to the principle of natural selection, members of a species that inherit certain adaptive characteristics are more likely to survive and propagate, thereby Quick Quiz B 1. Thorndike’s cats learned to solve the puzzle box problem (gradually/suddenly). 2. Based on his research with cats, Thorndike formulated his famous 216 Chapter 6 Operant Conditioning: Introduction passing that characteristic on to offspring. Thus, over many generations, the frequency of those adaptive characteristics within the population will increase and become well established. Similarly, according to the principle of operant conditioning, behaviors that lead to favorable outcomes are more likely to be repeated while those that do not lead to favorable outcomes are less likely to be repeated. Thus, operant conditioning is sort of a mini-evolution of an organism’s behaviors, in which behaviors that are adaptive (lead to favorable outcomes) become more frequent while behaviors that are nonadaptive (do!not lead to favorable outcomes) become less frequent. The operant conditioning process can be conceptualized as involving three components: (1) A response that produces a certain consequence (e.g.,!lever pressing produces a food pellet); (2) the consequence that serves to either increase or decrease the frequency (or probability) of the response that preceded it (e.g., the consequence of a food pellet increases the rat’s tendency to again press the lever); and (3) a discriminative stimulus that precedes the response and signals that a certain consequence is now available (e.g., a tone that signals to the rat that a lever press will now produce food). These components are examined in more detail in the following section. Operant Behavior An operant behavior is a class of emitted responses that result in certain consequences; these consequences then affect the future frequency (or probability) of those responses. Operant responses are sometimes simply called operants, and the probability or frequency of the response is often referred to as the strength of the behavior (in other words, the terms probability, frequency, and strength should be regarded as equivalent). Suppose, for example, that a rat presses a lever and receives a food pellet, with the result that it is more likely to press the lever in the future. Lever press → Food pellet The effect: the future frequency (or strength) of lever pressing increases. Or, Jonathan might tell a joke and receive a frown from the person he tells it to. He is now less likely to tell that person a joke in the future. Tell a joke → Person frowns The effect: the future frequency (or strength) of telling a joke decreases. In each case, the behavior in question (the lever pressing or the joke telling) is an operant response (or an “operant”) because its occurrence results in a certain consequence and that consequence affects the future frequency of the response. (Note that, for the sake of simplicity, the definitions and examples in this chapter will focus on whether the consequence increases or decreases the future frequency of a behavior. However, there are obvious limitations in how far that can progress, such that a consequence sometimes serves to simply maintain responding at a certain frequency. The manner in which certain patterns of reinforcement maintain a certain frequency of responding will especially be covered in our discussion of schedules of reinforcement in Chapter 7.) 6.2 Operant Conditioning 217 In contrast to classically conditioned behaviors, which are said to be elicited by stimuli (e.g., food elicits salivation), operant behaviors are technically said to be emitted by the organism (e.g., the rat emits lever pressing or the person emits the behavior of telling jokes). This wording is used to indicate that operant behavior appears to have a more voluntary, flexible quality to it compared to elicited behavior, which is generally more reflexive and automatic. (Does this mean that operant behavior actually is voluntary? Not necessarily. Insofar as such behavior comes to be controlled by the consequences that follow the behavior, it can be argued that the sense of voluntariness, or “freedom of choice,” that accompanies such behavior is merely an illusion [Skinner, 1953].) Operant behavior is typically defined as a class of responses, with all of the responses in that class capable of producing the consequence. For example, there are many ways a rat can press a lever for food: hard or soft, quick or slow, right paw or left paw. All of these responses are effective in depressing the lever and producing food; therefore, they all belong to the same class of responses known as “lever presses.” Similarly, Jonathan could tell many different jokes, and he could even tell the same joke in many different ways, all of which might produce a laugh. Defining operants in terms of classes has proven fruitful because it is easier to predict the occurrence of a class of responses than it is to predict the exact response that will be emitted at a particular point in time. For example, it is easier to predict that a hungry rat will press a lever to obtain food than it is to predict exactly how it will press the lever on any particular occasion. in that it is (more/less) mentalistic. 2. Operant conditioning is similar to the principle of natural selection in that 3. 4. 5. 6. 7. behaviors that are (adaptive/nonadaptive) tend to increase in frequency, while tend to decrease in frequency. The behaviors that are difference is that operant conditioning deals with changes at the level of a(n) (species/individual) while the principle of natural selection deals with changes at . the level of a(n) The process of operant conditioning involves the following three components: that produces a certain , (1) A r that serves to either increase or decrease the (2) a c that preceded it, and (3) a future likelihood of the stimulus that precedes the d is now available. and signals that a certain Classically conditioned behaviors are said to be e by the by the stimulus, while operant behaviors are said to be e organism. Operant responses are sometimes simply called . Operant behavior is usually defined as a(n) of responses rather than a specific response. Remember that the terms response and behavior are essentially equivalent; when referring to a however, behaviorists tend to use the term specific instance of a behavior. Quick Quiz C 1. Skinner’s definition of operant conditioning differs from Thorndike’s law of effect 218 Chapter 6 Operant Conditioning: Introduction Operant Consequences: Reinforcers and Punishers The second component of an operant conditioning procedure is the consequence that either increases (strengthens) or decreases (weakens) the!frequency of a behavior. Consequences that strengthen a behavior (make it more frequent) are called reinforcers, and consequences that weaken a behavior (make it less frequent) are called punishers. Thus, an event is a reinforcer if (1) it follows a behavior and (2) the future frequency of that behavior increases. Conversely, an event is a punisher if (1) it follows a behavior and (2) the future frequency of that behavior decreases. Diagrams of operant conditioning procedures generally use the following symbols. Reinforcers are usually given the symbol SR (which stands for reinforcing stimulus), and punishers are given the symbol SP (which stands for punishing stimulus). The operant response is given the symbol R. Using these abbreviations, a diagram of a procedure in which a lever press is reinforced by the delivery of a food pellet looks like this: Lever press (R) → Food pellet (SR) The food pellet is a reinforcer because it follows the lever press and increases the future probability of the rat pressing the lever. A diagram of Jonathan’s failed attempt at humor, in which a frown punished his behavior of telling jokes, looks like this: Tell a joke (R) → Person frowns (SP) The frown is a punisher because it follows the joke, and the future probability of Jonathan’s joke telling decreases. Note that, from a behavior analysis perspective, it is technically incorrect to say that a person or animal has been reinforced or punished; rather, it is the behavior that has been reinforced or punished. Only the behavior increases or decreases in frequency. (But not everyone is this strict or cautious in their wording, and it is not uncommon to read references to the animal being reinforced or punished. There is value, however, in emphasizing the effect of the consequence on behavior. If you want a child to stop doing something, should you tell them that their behavior displeases you or that they displease you? Similarly, when your roommate does something that bothers you, will it be more constructive to tell them that their behavior disturbs you or that they disturb you? Is it easier for people to change their behavior or to change who they are?) It is also important to differentiate the terms reinforcer and punisher from reinforcement and punishment. Reinforcer and punisher both refer to the specific consequence used to strengthen or weaken a behavior. In the previous examples, the food pellet is a reinforcer for lever pressing, and the frown is a punisher for joke telling. In contrast, the terms reinforcement and punishment usually refer to the process or procedure by which a certain consequence changes the strength of a behavior. Thus, the use of food to increase the strength of lever pressing is an example of reinforcement, while 6.2 Operant Conditioning the food itself is a reinforcer. Similarly, the process of frowning to discourage Jonathan from telling jokes is an example of punishment, while the frown itself is a punisher. In summary, the terms reinforcer and punisher refer to the actual consequences of the behavior; the terms reinforcement and punishment refer to the process or procedure of strengthening or weakening a behavior by instituting those consequences. Note, too, that reinforcers and punishers are formally defined entirely by their effect on behavior. For example, a laugh is a reinforcer for the behavior of joke telling only to the extent that joke telling then increases. If, for some reason, joke telling decreases as a result of the laugh (perhaps the person telling the joke delights in being disgusting and does not want people to find the joke funny), the laugh would by definition be a punisher. It is important to remember this, because events that on the surface seem like reinforcers or punishers do not always function in that manner. We encountered this notion in Chapter 2 in our discussion of the distinction between appetitive and aversive events. In similar fashion, a teacher might yell at their students for being disruptive, and as a result the students become more (not less) disruptive. Although the teacher is trying to punish the disruptive behavior, the yelling is actually having the opposite effect. By definition, therefore, the yelling is a reinforcer because it is causing the disruptive behavior to increase in frequency (perhaps because disruptive students find that other students admire them if they upset the teacher). Thus, the safest bet is to define consequences as reinforcers and punishers in relation to their effect on behavior and not in relation to how pleasant or unpleasant they seem. It is for this reason that many behaviorists prefer the term reinforcer rather than reward, the latter term being too strongly associated with events that are seemingly pleasant (e.g., affection, food, money). For example, the teacher’s yelling is hardly what anyone would call a reward, but technically speaking it is a reinforcer for the students’ disruptive behavior. Not all behaviorists are this strict in their terminology, however, and they sometimes use the terms reward and reinforcer interchangeably (e.g., Bandura, 1997; Herrnstein, 1997).3 Moreover, because students often find it helpful to think of consequences in terms of whether they are pleasant or unpleasant, we will sometimes make use of such terms in our discussion of consequences. In other words, to help you gain an initial grasp of this material, we will sometimes be rather informal in the terminology we use. (You should check with your professor, however, to determine if such informality will be acceptable in the course you are taking.) 3 Furthermore, some behaviorists use the term reward to refer to the effect of the consequence on the organism as opposed to behavior (Rachlin, 1991). For example, a dog biscuit can be both a reinforcer for the dog’s behavior of begging and a reward to the dog for having carried out such a behavior. Or, to put it differently, reinforcers strengthen our behaviors, while rewards make us happy. 219 220 Chapter 6 Operant Conditioning: Introduction Quick Quiz D Finally, you should be aware that punishment is not the only means of weakening a behavior. A response that has been strengthened through reinforcement can also be weakened by the withdrawal of reinforcement. The weakening of a behavior through the nonreinforcement of a previously reinforced behavior is known as extinction. For example, a child who has learned to whine for candy in the supermarket will eventually cease whining when behaving that way no longer results in candy. Likewise, someone who tells offensive jokes because of the outraged reaction they get from their roommates will eventually stop telling such jokes if the roommates stop reacting that way. Extinction is usually a much gentler process than punishment; one drawback to it, however, is that it is typically a much slower process. Extinction and the various issues associated with it are more fully discussed in Chapter 8. 1. Simply put, reinforcers are those consequences that s 2. 3. 4. 5. 6. 7. 8. 9. (or a behavior; that is, they (increase/decrease) its f ). p Punishers are those consequences that w (or a behavior; that is, they (increase/decrease) its f ). p A reinforcer is an event that (precedes/follows) a behavior and (increases/decreases) the future frequency of that behavior. A punisher is an event that (precedes/follows) a behavior and (increases/decreases) the future frequency of that behavior. The terms reinforcement and punishment refer to the pr whereby the future occurrence of a behavior is or pr strengthened or weakened by its consequences. Strengthening a roommate’s tendency toward cleanliness by thanking them when they clean the bathroom is an example of a (reinforcement/reinforcer), . while the thanks itself is a Eliminating a dog’s tendency to jump up on visitors by scolding it when it does so is an example of (punisher/punishment), while the scolding itself is a . Strictly speaking, reinforcers and punishers are defined entirely by their on behavior. For this reason, the term reinforcer is because the latter is too often preferred to the term closely associated with events that are regarded as pleasant or desirable, and such events do not always strengthen a behavior. When Moe stuck his finger in a light socket, he received an electric shock. As a result, he now sticks his finger in the light socket as often as possible. because the By definition, the electric shock was a behavior it followed has now increased in frequency. Each time Edna talked in class, her teacher immediately came over and gave her a hug. As a result, Edna no longer talks in class. By definition, the hug is because the behavior it follows has decreased in a frequency. 6.2 Operant Conditioning 10. When labeling an operant conditioning procedure, punishing consequences (which stands for ), while reinforcing (which stands for consequences (reinforcers) are given the symbol ). The operant response is given the . symbol 11. When we give a dog a treat for fetching a toy, are we attempting to reinforce (a) the behavior of fetching the toy or (b) the dog that fetched the toy? When we chastise a child for being rude, are we attempting to punish (a) the child who was rude or (b) the child’s rude behavior? 12. Weakening a behavior through the withdrawal of reinforcement for that . In general, this is a (slower/ behavior is known as faster) process than punishment. 13. Clayton stopped using the toaster after he received a shock while doing so. This is an example of (punishment/extinction). Manzar stopped using the toaster after . it no longer made good toast. This is an example of ( p u n i s h e rs ) a re g i v e n t h e s y m b o l Operant Antecedents: Discriminative Stimuli The operant response and its consequence are the most essential components of the operant conditioning procedure. in most circumstances, however, a third component can also be identified. When a behavior is consistently reinforced or punished in the presence of certain stimuli, those stimuli will begin to influence the occurrence of the behavior. For example, if lever pressing produces food only when a tone is sounding, the rat soon learns to press the lever only when it hears the tone. This situation can be diagrammed as follows: Tone (SD): Lever press (R) → Food pellet (SR) This sequence can be read as follows: in the presence of the tone, if the rat presses the lever, it will receive food. The tone is called a discriminative stimulus. Discriminative stimuli are traditionally given the symbol S D (pronounced “es-dee”). A discriminative stimulus (SD) is a stimulus in the presence of which responses are reinforced and in the absence of which they are not reinforced. In other words, a discriminative stimulus is a signal that indicates that a response will be followed by a reinforcer. Another example: if Susan always laughs at Jonathan’s jokes, then he is more likely to tell her a joke. The sight of Susan is an S D for Jonathan’s behavior of telling jokes. This can be diagrammed as follows: Susan (SD): Tell her a joke (R) → She laughs (SR) Discriminative stimuli are said to “set the occasion for” the behavior, meaning that the behavior is more likely to occur in the presence of 221 222 Chapter 6 Operant Conditioning: Introduction those stimuli. (This is similar to the concept of occasion setting in classical conditioning, as described in Chapter 4, so be sure that you can distinguish between the two concepts.) Discriminative stimuli do not elicit behavior in the manner of a CS or US in classical conditioning. For example, the tone does not automatically elicit a lever press; it merely increases the probability that a lever press will occur. Whether or not lever pressing occurs is still a function of its consequence (food), and the SD simply indicates that this consequence is now available. Similarly, the presence of Susan does not automatically elicit the behavior of joke telling in Jonathan; rather, he is simply more likely to tell a joke in her presence. Therefore, rather than saying that the SD elicits the behavior, we say that the person or animal emits the behavior in the presence of the S D. (Remember, it is only in classical conditioning that we say that the stimulus elicits the behavior. In operant conditioning, we say that the organism emits the behavior.) The discriminative stimulus, the operant behavior, and the reinforcer or punisher constitute what is known as the three-term contingency. The threeterm contingency can also be viewed as consisting of an antecedent event (an antecedent event is a preceding event), a behavior, and a consequence (which can be remembered by the initials ABC). Antecedent Behavior Consequence Susan (SD): Tell her a joke (R) → She laughs (SR) Tone (SD): Lever press (R) → Food pellet (SR) Another way of thinking about this sequence is that you notice something (Susan), do something (tell Susan a joke), and get something (Susan laughs at your joke) that then increases the likelihood of your repeating that response. Similarly, you notice that it is 7:00 ".#., you turn on the TV, and you get to enjoy your favorite sitcom. Or maybe your dog notices that you have popcorn, begs persistently, and is given some of the popcorn. Many students find this sequence easy to remember: notice something, do something, get something (although, as you will later read, the consequence in some cases involves losing or avoiding something rather than getting something). Analyzing a problem behavior in terms of the three-term contingency—the manner in which the antecedents and consequences affect the strength of the behavior—is considered a fundamental aspect of applied behavior analysis. So far, we have dealt only with stimuli that are associated with reinforcement. Stimuli can also be associated with punishment. A stimulus that signals that a response will be punished is called a discriminative stimulus for punishment (which is sometimes given the symbol SDp). For example, if a water bottle signals that meowing will result in being sprayed with water (rather than being fed), a cat will quickly learn to stop meowing whenever it sees the water bottle. 6.2 Operant Conditioning Water bottle (SDp): Meow (R) → Get sprayed (SP) Similarly, a motorist who receives a fine for speeding in the presence of a police car will soon learn to stop speeding in the presence of police cars. Police car (SDp): Speed (R) → Receive fine (SP) For the speeding motorist, the presence of a police car is a discriminative stimulus for punishment. A discriminative stimulus may also signal the occurrence of extinction; that is, it signals the nonavailability of a previously available reinforcer. If, for example, lever pressing is typically followed by the presentation of food, but only when a tone is sounding and not when a buzzer is sounding, then: Tone (SD): Lever press (R) → Food pellet (SR) Buzzer (S!): Lever press (R) → No food (—) The rat that is subjected to this procedure soon learns to press the lever only when the tone sounds and not when the buzzer sounds. The buzzer in this case is a discriminative stimulus for extinction, which is a stimulus that signals the absence of reinforcement. As you can read in the preceding example, the discriminative stimulus for extinction is typically given the symbol S$ (pronounced “es-delta”). As noted earlier, the process of extinction is more fully discussed in Chapter 8.4 Finally, you should be aware that processes of operant and classical conditioning overlap such that a particular stimulus can simultaneously act as both a discriminative stimulus and a conditioned stimulus. For example, consider a tone that serves as an SD for the operant behavior of lever pressing for food: Tone (SD): Lever press (R) → Food (SR) The tone is closely associated with food, and food, of course, elicits salivation. This means that during the course of our operant conditioning procedure, the tone will also become a conditioned stimulus (CS) that elicits salivation as a conditioned response (CR). Thus, if we ignore the lever pressing and concentrate just on the salivation, then what is happening is this: Tone (NS): Food (US) → Salivation (UR) Tone (CS) → Salivation (CR) Whether the tone should be considered an SD or a CS depends on the response to which one is referring. It is an SD with respect to the operant 4 The symbols for discriminative stimuli are not entirely standardized. Some textbooks use S+ (positive discriminative stimulus) to denote the discriminative stimulus for reinforcement and S– (negative discriminative stimulus) to denote the discriminative stimulus for extinction or punishment. 223 224 Chapter 6 Operant Conditioning: Introduction response of lever pressing and a CS with respect to the classically conditioned response of salivation. (Refer to Table 6.1 for a summary of the differences between classical and operant conditioning.) Table 6.1 Differences between operant and classical conditioning. Note that these are traditional differences. As you will read in Chapter 12, the distinction between classical and operant conditioning is sometimes less clear than what is depicted here. Classical Conditioning Operant Conditioning Behavior is generally seen as involuntary and inflexible. Behavior is said to be “elicited by the stimulus.” This type of conditioning usually involves innate patterns of behavior (URs). Behavior is a function of what comes before it; that is, the preceding stimulus is critical and the consequences are largely irrelevant. Behavior is generally seen as voluntary and flexible. Behavior is said to be “emitted by the organism.” This type of conditioning often does not involve innate patterns of behavior. Behavior is a function of what comes after it; that is, the consequences are critical and the preceding stimulus simply signals whether those consequences are available. Conditioning involves a stimulusresponse-stimulus (S–R–S) sequence. Conditioning involves a stimulusstimulus-response (S–S–R) sequence. Quick Quiz E In general, to determine if operant or classical conditioning is involved, the most important question to ask is whether the behavior is mostly a function of what precedes it (classical conditioning) or what might follow it (operant conditioning). 1. The operant conditioning procedure usually consists of three components: 2. 3. 4. 5. s , (2) an (1) A d response, and (3) a c . o A discriminative stimulus is usually indicated by the symbol . A discriminative stimulus is said to “ the for the behavior,” meaning that its presence makes the response (more/less) likely to occur. A discriminative stimulus (does/does not) elicit behavior; a conditioned stimulus (does/does not) elicit behavior. Using the appropriate symbols, label each component in the following three-term contingency (assume that the behavior will be strengthened): ): Answer phone ( ) → Conversation with friend ( ) Phone rings ( 6.3 Four Types of Contingencies 6. The three-term contingency can also be thought of as an ABC sequence, where , B stands for , and C stands A stands for . for 7. Another way of thinking about the three-term contingency is that you something, something, and something. 8. A stimulus in the presence of which a response is punished is called a for . . It can be given the symbol 9. A bell that signals the start of a round in a boxing match and therefore serves as an SD for the operant response of beginning to box may also serve as a(n) (SD/CS) for a fear response. This is an example of how the processes conditioning and of conditioning often overlap. 6.3 Four Types of Contingencies As described previously, there are two main types of consequences in operant conditioning: reinforcers and punishers. If the response is followed by a reinforcer, then we say that a contingency of reinforcement exists (meaning that the delivery of the reinforcer is contingent upon the response); if the response is followed by a punisher, we say that a contingency of punishment exists. However, contingencies of reinforcement and punishment can be further divided into two subtypes: positive and negative. This results in four basic types of contingencies (response–consequence relationships): positive reinforcement, negative reinforcement, positive punishment, and negative punishment. Because these are sometimes confusing to students, we will describe them in some detail. As you learned previously, reinforcement is a procedure that strengthens a behavior (increases the frequency or probability of a behavior), and punishment is a procedure that weakens a behavior (decreases the frequency or probability of a behavior). That part is pretty straightforward, but this next part can be tricky. When combined with the words reinforcement or punishment, the word positive means only that the behavior is followed by the presentation or addition of something (think of a + [positive] sign, which means “add”). Thus, the word positive, when combined with the terms reinforcement or punishment, does not mean good or pleasant; it means only that the response has resulted in something being added or presented. The event that is presented could either be pleasant (receiving a compliment) or unpleasant (getting yelled at). Similarly, the word negative, when combined with the words reinforcement or punishment, means only that the behavior is followed by the removal of something; that is, something is subtracted from the situation (think of 225 226 Chapter 6 Operant Conditioning: Introduction Quick Quiz F a!– [negative] sign, which means “subtract”). The word negative, therefore, in this context, does not mean bad or unpleasant; it means only that the response results in the removal of something. The something that is removed could be an event that is pleasant (your dessert is taken away) or unpleasant (the person stops yelling at you). To summarize, in the case of positive reinforcement and positive punishment, the word positive means only that the behavior has resulted in something being presented or added. In negative reinforcement and negative punishment, the word negative means only that the behavior has resulted in something being removed or subtracted. The word reinforcement, of course, means that the behavior will be more likely to occur in the future, and the word punishment means that the behavior will be less likely to occur in the future. Thus, to determine which type of contingency is involved in a particular instance, ask yourself the following two questions: (1) Does the consequence consist of something being presented or withdrawn? If the consequence consists of something being presented, then it is a positive contingency; if the consequence consists of something being withdrawn, then it is a negative contingency; (2) Does the consequence serve to strengthen or weaken future occurrences of the behavior? If it strengthens future occurrences of the behavior, then we are dealing with reinforcement; if it weakens future occurrences of the behavior, then we are dealing with punishment. Apply these two questions to situations that you encounter, and you will almost always (we will deal with a few exceptions later) be able to determine which of the four types of contingencies is involved. Finally, you will note in the next examples that we have used the generic symbols S R and S P for both the positive and negative versions of reinforcement and punishment. However, more specific symbols are sometimes used, such as S R+ for positive reinforcement, S R– for negative reinforcement, SP+ for positive punishment, and SP– for negative punishment. Your professor can let you know if they have any preferences in this regard. 1. The word positive, when combined with the word reinforcement or punishment, of means only that the behavior is followed by the something. The word negative, when combined with the word reinforcement or punishment, means only that the behavior is followed by the of something. 2. The word positive, when combined with the word reinforcement or punishment, (does/does not) mean that the consequence is good or pleasant. Similarly, the term negative, when combined with the word reinforcement or punishment, (does/does not) mean that the consequence is bad or unpleasant. 3. Within the context of reinforcement and punishment, positive refers to the (addition/subtraction) of something, and negative refers to the (addition/ subtraction) of something. 6.3 Four Types of Contingencies 4. Reinforcement results in a(n) (increase/decrease) in the future frequency of a behavior, whereas punishment results in a(n) (increase/decrease) in the future frequency of a behavior. Positive Reinforcement Positive reinforcement consists of the presentation of a stimulus (one that is usually considered pleasant or rewarding) following a response, which then leads to an increase in the future strength of that response. Loosely speaking, the behavior results in the delivery of something the person or animal likes, so they are more likely to behave that way in the future. Some of the earlier illustrations have been examples of positive reinforcement. The standard rat procedure in which lever pressing produces food is an example of positive reinforcement because the consequence of food leads to a future increase in lever pressing. It is reinforcement because the behavior increases in frequency, and it is positive reinforcement because the consequence involves the presentation of something—namely, food (which we would call a positive reinforcer). Here are other examples of positive reinforcement: Turn on TV (R) → Get entertainment (SR) Smile at person (R) → Person smiles back at you (SR) Order coffee (R) → Receive coffee (SR) Study diligently for quiz (R) → Obtain an excellent mark (SR) Compliment partner (R) → Receive a kiss (SR) Negative Reinforcement Negative reinforcement is the removal of a stimulus (one that is usually considered unpleasant or aversive) following a response, which then leads to an increase in the future strength of that response. Loosely speaking, the behavior results in the prevention or removal of something the person or animal hates, so they are more likely to behave that way in the future. For example, if by pressing a lever a rat terminates an electric shock that it is receiving, it will become more likely to press the lever the next time!it receives an electric shock. This is an example of reinforcement because the behavior increases in strength; it is negative reinforcement because!the!consequence consists of taking something away. Here are some additional examples: Open umbrella (R) → Escape rain (SR) Claim illness (R) → Avoid writing an exam (SR) 227 228 Chapter 6 Operant Conditioning: Introduction Take aspirin (R) → Eliminate headache (SR) Turn on the heater (R) → Escape the cold (SR) Quick Quiz G The last example is interesting because it illustrates how it is sometimes a matter of interpretation as to whether something is an example of negative reinforcement or positive reinforcement. Does the person turn on the heater to escape the cold (negative reinforcement) or to obtain warmth (positive reinforcement)? Either interpretation would be correct. Negative reinforcement involves two types of behavior: escape and avoidance. Escape behavior results in the termination (stopping) of an aversive stimulus. In the example of the person getting rained on, by opening the umbrella the person is able to stop this from happening. Likewise, taking aspirin removes a headache, and turning on the heater allows one to escape the cold. Avoidance is similar to escape except that avoidance behavior occurs before the aversive stimulus is presented and therefore prevents its delivery. For example, by opening the umbrella before stepping out into the rain, the person avoids getting rained on. And by pretending to be ill, a student avoids having to take an exam. Escape and avoidance are discussed in more detail in Chapter 9. 1. When you reached toward the dog, it nipped at your hand. You quickly pulled 2. 3. 4. 5. your hand back. As a result, it now nips at your hand whenever you reach toward it. The consequence for the dog’s behavior of nipping consisted of the (presentation/removal) of a stimulus (namely, your hand), and its behavior of nipping subsequently (increased/decreased) in frequency; therefore, this is an reinforcement. example of When the dog sat at your feet and whined during breakfast one morning, you fed it. As a result, it sat at your feet and whined during breakfast the next morning. The consequence for the dog’s whining consisted of the (presentation/ removal) of a stimulus, and its behavior of whining subsequently (increased/ decreased) in frequency; therefore, this is an example of reinforcement. Kayla cries while saying, “Jo, I don’t feel as though you love me.” Jo gives Kayla a big hug, saying, “That’s not true, dear, I love you very much.” If Jo’s hug is a reinforcer, Kayla is (more/less) likely to cry the next time she feels insecure about their relationship. More specifically, this is an example of reinforcement of Kayla’s crying behavior. With respect to escape and avoidance, an response is one that terminates an aversive stimulus, while an response is one that prevents an aversive stimulus from occurring. Both types of responses are maintained by (positive/negative) reinforcement. Turning down the heat because you are too hot is an example of an (escape/ avoidance) response; turning it down before you become too hot is an example of an (escape/avoidance) response. 6.3 Four Types of Contingencies Positive Punishment Positive punishment consists of the presentation of a stimulus (one that is usually considered unpleasant or aversive) following a response, which then leads to a decrease in the future strength of that response. Loosely speaking, the behavior results in the delivery of something the person or animal hates, so the subject is less likely to behave that way in the future. For example, when a rat receives a shock after pressing a lever, it no longer presses the lever in the future. This is an example of punishment because the behavior decreases in strength, and it is positive punishment because the consequence involves the presentation of something (i.e., shock). Consider some further examples of positive punishment: Talk back to the boss (R) → Get reprimanded (SP) Swat at a wasp (R) → Get stung (SP) Meow constantly (R) → Get sprayed with water (SP) In each case, the behavior is followed by the presentation of an aversive stimulus, with the result that there is a decrease in the future probability of the behavior. People frequently confuse positive punishment with negative reinforcement. One reason for this is the fact that many behaviorists use the term negative reinforcer to refer to an aversive (unpleasant) stimulus and the term positive reinforcer to refer to an appetitive (pleasant) stimulus. Unfortunately, people with less knowledge of the field have then assumed that the presentation of a negative reinforcer is an instance of negative reinforcement, which it is not. Within the framework presented here, it is instead an instance of positive punishment. Negative Punishment Negative punishment consists of the removal of a stimulus (one that is usually considered pleasant or rewarding) following a response, which then leads to a decrease in the future strength of that response. Loosely speaking, the behavior results in the removal of something the person or animal likes, so the subject is less likely to behave that way in the future. Here are some examples of negative punishment: Stay out past curfew (R) → Lose car privileges (SP) Argue with boss (R) → Lose job (SP) Play with food (R) → Lose dessert (SP) Tease sister (R) → Sent to room (loss of social contact) (SP) 229 230 Chapter 6 Operant Conditioning: Introduction In each case, it is punishment because the behavior is less likely to occur in the future, and it is negative punishment because the consequence consists of removing something. The last example is known as “time-out” and is employed by many parents as a replacement for spanking. Removal of social contact is usually one consequence of such a procedure; more generally, however, the child loses the opportunity to receive any type of positive reinforcer during the time-out interval. Children usually find such situations unpleasant, with the result that even very brief time-outs can be effective. Consider another example of negative punishment: Jonathan’s girlfriend, who is quite jealous, completely ignored him (withdrew her attention from him) when she observed him having a conversation with another woman at a party. As a result, he stopped talking to the other women at the party. Jonathan talks to other women (R) → His girlfriend ignores him (SP) Jonathan’s behavior of talking to other women at parties was negatively punished. It is punishment in that the frequency with which he talked to other women at the party declined, and it is negative punishment because the consequence that produced that decline was the withdrawal of his girlfriend’s attention. Question: In this scenario, Jonathan’s behavior has been negatively punished. But what contingencies are operating on the girlfriend’s behavior? When she ignored him, he stopped talking to other women at the party. Given that this occurred, she might ignore him at future parties if she again sees him talking to other women. If so, her behavior has been negatively reinforced by the fact that it was effective in getting him to stop doing something she disliked. If we diagram this interaction from the perspective of each person, we get the following: For Jonathan: I talk to other women (R) → My girlfriend ignores me (SP) For his girlfriend: I ignore Jonathan (R) → He stops talking to other women (SR) As you can see, a reduction in one person’s behavior as a result of punishment can negatively reinforce the behavior of the person who implemented the punishment. This is the reason we are so often enticed to use punishment: punishment is often successful in immediately getting a person to stop behaving in ways that we dislike. That success then reinforces our tendency to use punishment in the future, which of course can create major problems 6.3 Four Types of Contingencies in the long run. We discuss the uses and abuses of punishment more fully in Chapter 9.5 Many people mistakenly equate behaviorism with the use of punishment. It is important to recognize that behaviorists actually emphasize the use of positive reinforcement. Indeed, Skinner (1953) believed that many societal problems can be traced to the overuse of punishment as well as negative reinforcement. For example, teachers too often control their students by attempting to punish maladaptive behavior rather than by positively reinforcing adaptive behavior. Moreover, the educational system in general is designed in such a way that students too often study to avoid failure (a negative reinforcer) rather than to obtain knowledge (a positive reinforcer). As a result, schooling is often more onerous and less effective than it could be. Similarly, in interpersonal relationships, people too often attempt to change each other’s behavior through the use of aversive consequences, such as complaining, when positive reinforcement for appropriate behavior might work just as well or better. Jelena, for example, says that Samira forgets to call whenever she is going to be late, even though Jelena often complains about it. Perhaps a more effective approach would be for Jelena to express her appreciation when Samira does call. Of course, this is not to suggest that Samira’s behavior is somehow Jelena’s fault (or that expressing appreciation will necessarily work). It is just that we are often unaware of the extent to which our behavior affects the behavior of others, especially when it involves punishment. That is why it is so valuable to learn about the processes of reinforcement and punishment; once we become aware of them, we are then more likely to react in ways that are potentially more beneficial and less harmful. Furthermore, although many people believe that the key to a great relationship is open communication, research has shown that a much more important element is the ratio of positive (pleasant) interactions to negative (aversive) interactions. In fact, one of the best predictors of a successful marriage is when the positives outweigh the negatives by a ratio of about five to one (Gottman, 1994). Even volatile relationships, in which there seems to be an enormous amount of bickering, can thrive if the number of positive exchanges, such as teasing, hugging, and praising, greatly outweigh the number of negative exchanges. To help strengthen your understanding of the four types of contingencies— positive reinforcement, negative reinforcement, positive punishment, and negative punishment—and deal with examples that are potentially confusing, refer to “Operant Contingencies: Tricky Examples and Common Confusions” in the And Furthermore box. 5 Note that the labels for the two types of punishment are not standardized. For example, positive and negative punishment are sometimes called Type 1 and Type 2 punishment. 231 Quick Quiz H 232 Chapter 6 Operant Conditioning: Introduction 1. When Sasha was teasing the dog, it bit her. As a result, she no longer teases 2. 3. 4. 5. the dog. The consequence for Sasha’s behavior of teasing the dog was the (presentation/removal) of a stimulus, and the teasing behavior subsequently (increased/decreased) in frequency; therefore, this is an example of . Whenever Sasha pulled the dog’s tail, the dog left and went into another room. As a result, Sasha now pulls the dog’s tail less often whenever it is with her. The consequence for pulling the dog’s tail was the (presentation/removal) of a stimulus, and the behavior of pulling the dog’s tail subsequently (increased/ decreased) in frequency; therefore, this is an example of . When Alex burped in public, Stephanie got angry with him. Alex now burps quite often when he is around Stephanie. The consequence for burping was of a stimulus, and the behavior of burping the in frequency; therefore, this is an example subsequently . of When Alex held the door open for Stephanie, she made a big fuss over what a gentleman he was becoming. Alex no longer holds the door open for her. The consequence for holding open the door was the of a stimulus, and the behavior of holding open the door subsequently in frequency; therefore, this is an example of . When Tenzing shared his toys with his brother, his mother stopped criticizing him. Tenzing now shares his toys with his brother quite often. The consequence for of a stimulus, and the behavior sharing the toys was the in frequency; therefore, of sharing the toys subsequently . this is an example of And Furthermore Operant Contingencies: Tricky Examples and Common Confusions After learning about the four types of contingencies, students are sometimes dismayed when they encounter examples that suddenly confuse them. When this happens, the contingency has often been worded in an unusual way. For example, suppose a parent tells a child that “if you do not clean your room, you won’t get any dessert.” What type of contingency is this statement referring to? (Think about this for a while before reading further.) To begin with, it sounds like a negative contingency because the consequence seems to involve the threatened loss of something—namely, the dessert. It also sounds like reinforcement because the goal is to increase the probability of a certain behavior—cleaning the room. We might therefore conclude that this is an example of negative reinforcement. But does this make sense? In everyday terms, negative reinforcement involves strengthening a behavior by removing something that the person dislikes, while here we are talking about removing something that the person likes. So what’s going on here? To clarify situations like this, it helps to reword the example in terms of the occurrence of a behavior rather than its nonoccurrence, because in reality one can only reinforce or 6.3 Four Types of Contingencies 233 punish the occurrence of a behavior and not its nonoccurrence. By rewording it in this way, and depending on which behavior we focus upon, this example can be interpreted as fitting one of two types of contingencies. On one hand, if the target behavior is cleaning the room, then this can be viewed as an example of positive reinforcement: if the child cleans their room, they can have dessert. On the other hand, if the target behavior is “doing something other than cleaning the room” (or something other than following their parent’s instructions), then the parent’s statement can be interpreted as referring to negative punishment: if the child does something other than clean their room, they will not get dessert. Thus, all behaviors other than room cleaning, such as watching television, will result in the loss of dessert. In fact, to the extent that the parent made the request in a threatening manner, this is probably what they intended. But note how the request could easily have been worded in the form of positive reinforcement—“If you clean your room, you can have some dessert”—and how much more agreeable that sounds. Unfortunately, parents too often choose the unpleasant version, especially when they are frustrated or angry, which in turn helps to create a decidedly unpleasant atmosphere in the household. Another source of confusion happens when students forget that contingencies of reinforcement and punishment refer to increases or decreases in the future occurrence of a behavior and not its present occurrence. Consider the following example: Jonah is throwing a temper tantrum because he’s bored with doing homework and wants to play on the iPad. So his mother gives him the iPad, at which point he stops throwing a temper tantrum. What type of contingency does this represent? At first glance, it could almost be interpreted as an example of positive punishment since (1) his behavior of throwing a tantrum has decreased, and (2) the consequence to that behavior consisted of something being presented (he was given an iPad). But, as with the tricky example discussed previously, this doesn’t make sense in that the iPad is something that he apparently wants. The problem here is that we are observing a decrease only in the present occurrence of the behavior and not its future occurrence, whereas the processes of reinforcement and punishment refer to the effect of a consequence on the future occurrence of the behavior. Thus, with respect to Jonah, the question is whether giving him an iPad to stop his temper tantrum will result in an increase or decrease in his future likelihood of throwing a temper tantrum the next time he gets bored. If, as seems likely, it results in an increase in the future occurrence of throwing a tantrum, then this would be an example of positive reinforcement. Finally, be aware that operant behaviors that are well learned can sometimes occur so automatically as to seem like reflexes. For example, most drivers, when they see a red light, will “automatically” step on the brakes. Because the act of stepping on the brakes seems almost reflexive, some students assume that this is an example of classical conditioning, with the red light being a CS that elicits the CR of stepping on the brakes. But this is incorrect. The effective consequence that motivates stepping on the brakes is actually the avoidance of getting a traffic ticket (or getting into an accident) with the red light being the discriminative stimulus for this contingency. This is therefore an example of negative reinforcement: Red light (SD): Step on brake (R) → Avoid getting a traffic ticket (SR) Operant behaviors that are so well learned that they seem to occur automatically are what we typically call habits and will be further discussed in Chapter 9. 234 Chapter 6 Operant Conditioning: Introduction 6.4 Positive Reinforcement: Further Distinctions Because behaviorists so strongly emphasize positive reinforcement, let us have a closer look at this type of contingency. Immediate Versus Delayed Reinforcement A reinforcer can be presented either immediately after a behavior occurs or following some delay. In general, the more immediate the reinforcer, the stronger its effect on the behavior. Suppose, for example, that you wish to reinforce a child’s behavior of playing quietly by giving him a treat. The treat should ideally be given while the quiet period is still in progress. If, instead, you deliver the treat several minutes later, while he is engaged in some other behavior (e.g., banging a stick on the floor), you might inadvertently reinforce that behavior rather than the one you wish to reinforce. The weak effect of delayed reinforcers on behavior accounts for some major difficulties in life. Do you find it tough to stick to a diet or an exercise regime? This is mainly because the benefits of exercise and proper eating are delayed and therefore weak, whereas the enjoyable effects of alternate activities, such as watching television and drinking a soda, are immediate and therefore powerful. Similarly, have you ever promised yourself that you would study all weekend, only to find that you wasted your time watching television and going out with friends? The immediate reinforcement associated with these recreational activities effectively outweighed the delayed reinforcement associated with studying. Of course, what we are talking about here is the issue of self-control, a topic that is more fully discussed in Chapter 10 (and in the Appendix). The importance of immediate reinforcement is so profound that some behaviorists (e.g., Malott, 1989; Malott & Suarez, 2004) argue that a delayed reinforcer does not, on its own, actually function as a “reinforcer.” They point to experimental evidence indicating that delaying a reinforcer by even 30 seconds or so can greatly reduce its effectiveness (e.g., Grice, 1948; Keesey, 1964; refer also to J. Williams, 1973). This finding suggests that delayed reinforcers, to the extent that they are effective, may function by a different mechanism from immediate reinforcement, especially in humans. Thus, receiving a good mark on that essay you wrote last week does not reinforce the behavior of essay writing in the same way that immediately receiving a food pellet reinforces a rat’s tendency to press a lever (or immediately seeing your mother’s smile reinforces your tendency to give her compliments). Rather, in the case of humans, behaviors that appear to be strengthened by delayed reinforcers are often under the control of rules or instructions that we have received from others or generated for ourselves. These rules or instructions describe to us the 6.4 Positive Reinforcement: Further Distinctions 235 delayed consequences that can result from a behavior (e.g., “Gee, if I work on that essay tonight, I will probably get a good grade on it next week”), thereby bridging the gap between the behavior and the consequence. In this text, for simplicity, we will ignore some of the complexities associated with the issue of rules and delayed reinforcement, though we will briefly discuss rule-governed behavior in Chapter 11. For the present purposes, it is sufficient to note that delayed reinforcement is usually much less effective than immediate reinforcement. This also makes clear the crucial importance of immediate reinforcement when interacting with young children (and animals) who have little or no language capacity, because the use of instructions is essentially dependent on language. the reinforcer, the stronger its effect on behavior. 2. It is sometimes difficult for students to study in that the reinforcers for studying and therefore (weak/strong), whereas the reinforcers are for alternative activities, like checking social media, are . and therefore 3. It has been suggested that delayed reinforcers (do/do not) function in the same manner as immediate reinforcers. Rather, the effectiveness of delayed reinforcers or in humans is largely dependent on the use of i to bridge the gap between the behavior and the r consequence. Primary and Secondary Reinforcers A primary reinforcer (also called an unconditioned reinforcer) is an event that is innately reinforcing; that is, it is an unlearned reinforcer. Loosely speaking, primary reinforcers are things we are born to like rather than learn to like, and we therefore have an innate tendency to find those events to be reinforcing. Examples of primary reinforcers are food, water, proper temperature (neither too hot nor too cold), and sexual contact with a desired partner. Many primary reinforcers are associated with basic physiological needs, and their effectiveness is closely tied to a state of deprivation. For example, food is a highly effective reinforcer when we are food deprived and hungry but not when we are satiated. Some primary reinforcers, however, do not seem to be associated with a physiological state of deprivation. An animal (or person) cooped up in a boring environment will likely find access to a more stimulating environment highly reinforcing and will perform a response such as lever pressing (or driving to the mall) to gain such access. In cases such as this, the deprivation is, in a sense, psychological rather than physiological. Quick Quiz I 1. In general, the more 236 Chapter 6 Operant Conditioning: Introduction A secondary reinforcer (also called a conditioned reinforcer) is an event that is reinforcing because it has been associated with some other reinforcer. Loosely speaking, secondary reinforcers are those events that we have learned to like because they have become associated with other things we like. Much of our behavior is directed toward obtaining secondary reinforcers, such as good grades, fine clothes, and a nice car. Because of our experiences with these events, they can function as effective reinforcers for our current behavior. For example, if good grades in school are consistently associated with praise from others, then the good grades themselves might serve as reinforcers for studying even in the absence of praise. And just seeing a teacher who once provided you with lots of praise may be an effective reinforcer for the behavior of visiting them many years later. Conditioned stimuli (CSs) that have been classically conditioned using appetitive unconditioned stimuli (USs) can also function as secondary reinforcers. For example, suppose that the sound of a metronome has been paired with food to produce a classically conditioned response of salivation: Metronome (NS): Food (US) → Salivation (UR) Metronome (CS) → Salivation (CR) The metronome, through its association with food, can now be used as a secondary reinforcer for an operant response such as lever pressing: Lever press (R) → Metronome (SR) The animal essentially seeks out the metronome because of its pleasant associations. Similarly, we might frequently seek out certain music that has been closely associated with a romantic episode in our life because of its pleasant associations. (This might partially explain why each generation seems to prefer the music of their youth; it’s associated with the good times they once had, or are now having, in their youth.) Discriminative stimuli associated with reinforcers can likewise function as secondary reinforcers. Consider a tone that has served as an SD signaling the availability of food for lever pressing: Tone: (SD): Lever press (R) → Food (SR) This tone can now be used as a secondary reinforcer for some other behavior, such as running in a wheel: Run in wheel (R) → Tone (SR) An important type of secondary reinforcer is known as a generalized reinforcer. A generalized reinforcer (also known as a generalized secondary reinforcer) is a type of secondary reinforcer that has been associated with several other reinforcers. For example, money is a powerful generalized reinforcer for humans because it is associated with an almost unlimited array of other reinforcers, including food, clothing, furnishings, entertainment, and even dates (insofar as money will likely increase our attractiveness to 6.4 Positive Reinforcement: Further Distinctions others). In fact, money can become such a powerful reinforcer that some people would rather just have the money than the things it can buy. Social attention, too, is a highly effective generalized reinforcer, especially for young children (though some aspects of it, such as touching, are probably also primary reinforcers). Attention from caretakers is usually associated with a host of good things, such as food and play and comfort, with the result that attention by itself can become a powerful reinforcer. It is so powerful that some children will even misbehave to get someone to pay attention to them, even if the attention consists of the person being angry with them. In fact, this is one of the ways in which punishment can backfire; what is intended to be a punisher actually serves as a positive reinforcer in the form of attention. Generalized reinforcers are often used in behavior modification programs. In a “token economy,” tokens are used in institutional settings—such as mental health hospitals, prisons, or classrooms for children with behavioral problems—to increase the frequency of desirable behaviors, such as completing an assigned task, dressing appropriately, or behaving sociably. Attendants deliver the tokens immediately following the occurrence of the behavior. These tokens can later be exchanged for “backup reinforcers,” such as treats, fun activities, or television time. In essence, just as the opportunity to earn money motivates many of us to behave appropriately, so too does the opportunity to earn a token motivate the individuals to behave appropriately. (Refer to Miltenberger, 2012, for a more detailed discussion of token economies.) Note that an event can function as both a primary reinforcer and a secondary reinforcer. A Thanksgiving dinner, for example, can be both a primary reinforcer, in the sense of providing food, and a secondary reinforcer due to its association with a beloved grandmother who prepared many similar dinners in your childhood. Finally, just as stimuli that are associated with reinforcement can become secondary reinforcers, so can behaviors that are associated with reinforcement. For example, children who are consistently praised for helping others might eventually find the behavior of helping others to be reinforcing in and of itself. They will then help others not to receive praise but because they “like to help.” We would then describe such children as having an altruistic nature. By a similar mechanism, in what is known as learned industriousness theory (Eisenberger, 1992), even hard work can become a secondary reinforcer. Rats, for example, that have received reinforcement for emitting forceful lever presses will afterward run faster down an alleyway to obtain food (Eisenberger, Carlson, Guile, & Shapiro, 1979). And students who have received reinforcement for solving complex math problems will subsequently write essays of higher quality (Eisenberger, Masterson, & McDermitt, 1982). Experiments have also confirmed the opposite: rats and humans that have received reinforcers for displaying low effort on a task will often perform poorly on subsequent tasks. (This is something to think about if you have a tendency to always take the easy way out.) 237 Quick Quiz J 238 Chapter 6 Operant Conditioning: Introduction 1. Events that are innately reinforcing are called p reinforcers. They are also called un reinforcers. 2. Events that become reinforcers through their association with other reinforcers 3. 4. 5. 6. 7. reinforcers. They are also called are called s reinforcers. c For most people, candy would be an example of a reinforcer, while a coupon that is used to purchase candy would be an example of reinforcer. a A (CS/US) that has been associated with an appetitive (CS/US) can serve as a secondary reinforcer for an operant response. As well, a stimulus that serves for an operant response can also serve as a as a(n) secondary reinforcer for some other response. A generalized reinforcer (or generalized reinforcer) is a secondary reinforcer that has been associated with . Two generalized reinforcers that have strong effects on human behavior are and . Behavior modification programs in institutional settings often utilize generalized . This type of arrangement is reinforcers in the form of t ec . known as a t Intrinsic and Extrinsic Reinforcement In the preceding discussion, we noted that operant behavior itself can sometimes be reinforcing. Such a behavior is said to be intrinsically reinforcing or motivating. Thus, intrinsic reinforcement is reinforcement provided by the mere act of performing the behavior. We go ice skating because it is invigorating, we spend time with friends because we like their company, and we work hard at something partly because hard work has, through experience, become enjoyable (though we are not always convinced about that one). Animals, too, sometimes engage in activities for their own sake. In some of the earliest research on intrinsic motivation, it was found that with no additional incentive, monkeys repeatedly solved mechanical puzzles (Harlow, Harlow, & Meyer, 1950). Unfortunately, many activities are not intrinsically reinforcing and instead require additional incentives to ensure their performance. Extrinsic reinforcement is the reinforcement provided by some consequence that is external to, or additional to, the behavior (with the consequence itself known as an extrinsic reinforcer). For example, perhaps you are reading this text solely because of an upcoming exam. Passing the exam is the extrinsic reinforcer that is motivating your behavior. Other examples of extrinsically motivated behaviors are working for money, exercising to become muscular, and dating an attractive individual merely to enhance your reputation. Unfortunately, the distinction between intrinsic and extrinsic reinforcers is not always clear. For example, is candy an intrinsic or extrinsic reinforcer? In one sense, candy seems like an intrinsic reinforcer because eating it is an enjoyable activity; yet the candy exists external to the behavior that is being reinforced. In such cases, it often helps to focus on the behavior that is being strengthened. Imagine, for example, that we offer candy to a child to strengthen 6.4 Positive Reinforcement: Further Distinctions the behavior of being quiet in the supermarket. The candy is clearly an extrinsic reinforcer for the behavior of being quiet, but with respect to the behavior of eating candy, the candy is the critical component in an intrinsically reinforcing activity. In any case, do not fret too much if you encounter an example that seems confusing. The important thing is to be able to distinguish situations in which the motivation is clearly intrinsic (taking a bath for the pleasure of it) from those in which the motivation is clearly extrinsic (taking a bath so that your date this evening will be impressed with how nice you smell). Question: What happens if you are given an extrinsic reinforcer for an activity that is already intrinsically reinforcing? Say, for example, that you love ice skating and are fortunate enough to be hired each weekend to skate around a public rink while wearing a new line of sportswear. Will the experience of receiving payment for skating increase, decrease, or have no effect on your subsequent enjoyment of the activity? Although you might think that it would increase your enjoyment of skating (since the activity is not only enjoyable but also associated with money), some research indicates that experiences like this can decrease intrinsic interest. For example, Lepper, Green, and Nisbett (1973) found that children who enjoyed drawing with Magic Markers became less interested following a session in which they had been promised, and then received, a “good player” award for drawing with the markers. In contrast, children who did not receive an award or who received the award unexpectedly after playing with the markers did not show a loss of interest. Similar results were reported by other investigators (e.g., Deci & Ryan, 1985). However, some researchers found that extrinsic rewards have no effect on intrinsic interest (e.g., Amabile, Hennessey, & Grossman, 1986) or actually produce an increase in intrinsic interest (e.g., Harackiewicz, Manderlink, & Sansone, 1984). Unfortunately, despite these mixed findings, it is the damaging effects of extrinsic rewards on intrinsic motivation that are often presented to the public (e.g., Kohn, 2018). But is this a fair assessment of the evidence? Are the harmful effects of reinforcement the rule or the exception? Cameron and Pierce (1994) attempted to answer this question by conducting a meta-analysis of 96 well-controlled experiments that examined the effects of extrinsic rewards on intrinsic motivation. (A meta-analysis is a statistical procedure that combines the results of several separate studies, thereby producing a more reliable overall assessment of the variable being studied.) The meta-analysis by Cameron and Pierce indicated that extrinsic rewards usually have little or no effect on intrinsic motivation. Extrinsic rewards can occasionally undermine intrinsic motivation, but only when the reward is expected (i.e., the person has been instructed beforehand that they will receive a reward), the reward is tangible (e.g., it consists of money rather than praise), or the reward is given for simply performing the activity (and not for how well it is performed). They also found that verbal rewards, such as praise, often produce an increase in intrinsic motivation, as do tangible rewards given for high-quality performance. Cameron and Pierce (1994) concluded that extrinsic rewards can be safely applied in most circumstances and that the limited circumstances in which they decrease intrinsic motivation are easily avoided. Bandura (1997) likewise has argued that the dangers of extrinsic rewards on intrinsic motivation have been greatly overstated. 239 240 Chapter 6 Operant Conditioning: Introduction Not surprisingly, these findings by Cameron and Pierce (1994; also Eisenberger & Cameron, 1996) resulted in a great deal of controversy. Proponents of the harmful effects of extrinsic rewards soon countered with their own meta-analysis that seemed to support the notion that extrinsic reinforcers are generally harmful (e.g., Deci, Koestner, & Ryan, 2001a, 2001b). Cameron and colleagues, in turn, countered with yet further evidence that seemed to support their conclusion that such concerns are overblown (Cameron, Banko, & Pierce, 2001; Cameron & Pierce, 2002; Cameron, Banko, Pierce, & Gear, 2005). However, a more recent meta-analysis by Cerasoli et al. (2014) came to the conclusion that intrinsic and extrinsic reinforcers are not necessarily in conflict with each other and that both make important contributions to performance. It is also worth noting that Skinner himself recognized the value of intrinsic reinforcement, arguing for example that in a well-designed learning program, the act of learning and making progress should itself be highly reinforcing (Skinner, 1969b). (For those wishing to further explore this topic, especially regarding its implications for teachers, read Kohn, 2018, for what is probably the most strident criticism of the use of extrinsic rewards—even the use of praise is severely criticized—and Akin-Little & Little, 2019, and Scott & Landrum, 2020, for strident support of the use of extrinsic rewards.)6 (Refer also to “Positive Reinforcement of Artistic Appreciation” in the And Furthermore box.) And Furthermore Positive Reinforcement of Artistic Appreciation B.!F. Skinner (1983) once described how two students used positive reinforcement to instill in their new roommate an appreciation of modern art. These students had several items of modern art in their apartment, but the roommate had little interest in them and was instead proceeding to “change the character” of the space. As a counterploy, the students would pay attention to the roommate only when they saw him looking at one of the art works. They also threw a party and arranged for an attractive young woman to engage him in a discussion about modern art. They then arranged for him to receive announcements from local art galleries about upcoming art shows. After about a month, the roommate himself suggested attending a local art museum. Interestingly, while there, he just “happened” to find a five-dollar bill lying at his feet while he was looking at a painting. According to Skinner, “It was not long before [the two students] came again in great excitement—to show me his first painting” (p. 48). 6 It is also the case that a contingency that seems to involve positive reinforcement might in reality be more aversive in nature and an example of negative reinforcement. A catcher for a Major League Baseball team once told one of the authors of this text that he hated baseball because there were so many young players trying to replace him. The consequence that now motivated him was apparently the threatened loss of a good salary if he played poorly. According to Skinner (1987), human behavior is too often controlled by such negative consequences—working to avoid the loss of a paycheck and studying to avoid failure (especially prevalent in students who procrastinate until they are in danger of failing). It is therefore not surprising that these activities often seem less than intrinsically interesting. 6.4 Positive Reinforcement: Further Distinctions motivated activity is one in which the activity motivated activity is one in is itself reinforcing; a(n) which the reinforcer consists of some type of additional consequence that is external to the activity. 2. Running to lose weight is an example of a(n) motivated activity; running because it “feels good” is an example of a(n) motivated activity. 3. In their meta-analysis of relevant research, Cameron and Pierce (1994) found that extrinsic rewards decrease intrinsic motivation only when they are (expected/ unexpected), (tangible/verbal), or given for (performing well/merely engaging in the behavior). 4. They also found that extrinsic rewards generally increased intrinsic motivation when the rewards were (tangible/verbal), and that tangible rewards increased intrinsic motivation when they were delivered contingent upon (high/low)quality performance. Quick Quiz K 1. A(n) 241 Study Tip: Students often complain that they find it difficult to study, or even go to lectures, because their courses are so boring. Often, these are students who are doing the minimal amount of work to get by (if that). But the question can be asked: Are they doing minimal work because their courses are boring, or are their courses boring because they’re doing minimal work? In support of the latter possibility, students who, for whatever reason, suddenly decide to pull up their grades—perhaps because they’ve decided they want to do a professional or graduate degree—will sometimes comment that they didn’t realize how interesting their courses (or at least some of their courses) can be when they really apply themselves. This change in perception is, of course, in keeping with the notion that being rewarded for highquality performance can increase one’s intrinsic interest in a task. So if you tend to be a “minimalist” student who finds college courses boring, you might consider really applying yourself to your studies as part of the solution. Natural and Contrived Reinforcers The distinction between intrinsic and extrinsic reinforcers is closely related to the distinction between natural and contrived reinforcers. Natural reinforcers are reinforcers that are typically provided for a certain behavior; that is, they are an expected consequence of the behavior within that setting. Money is a natural consequence of selling merchandise; gold medals are a natural consequence of hard training and a great performance. Contrived (or artificial) reinforcers are reinforcers that have been deliberately arranged to modify a behavior; they are not a typical consequence of the behavior within that setting. For example, although television is the natural reinforcer for the behavior of turning on the television, it is a contrived reinforcer for the behavior of, say, accomplishing a certain amount of studying. In the latter case, we have created a contrived contingency in an attempt to modify the person’s study behavior. 242 Chapter 6 Operant Conditioning: Introduction Quick Quiz L Although contrived reinforcers are often seen as a hallmark of behaviorism, behaviorists strive to use natural reinforcers whenever possible (Sulzer-Azaroff & Mayer, 1991). When contrived reinforcers are used, the ultimate intention is to let the “natural contingencies” eventually take over if at all possible. For example, although we might initially use tokens to motivate a patient with schizophrenia to socialize with others, our hope is that the behavior will soon become “trapped” by the natural consequences of socializing (e.g., smiles and pleasant comments from others) such that the tokens can eventually be withdrawn. Similarly, although we might initially use praise to increase the frequency with which a child reads, the natural (and intrinsic) reinforcers associated with reading will hopefully take over so that the child will begin reading even in the absence of praise. It is important to remember that natural contingencies tend to produce more efficient behavior patterns than do contrived contingencies (Skinner, 1987). Although a coach might use praise to reinforce correct throwing actions by a young quarterback, the most important factor in producing correct throws will be the natural consequence of where the ball goes. To distinguish between intrinsic versus extrinsic reinforcers and natural versus contrived reinforcers, just remember that the former is concerned with the extent to which the behavior itself is reinforcing while the latter is concerned with the extent to which a reinforcer has been artificially imposed so as to manipulate a behavior. Note, too, that the extent to which a reinforcer has been artificially imposed is not always clear; hence, it is easy to find examples in which it is ambiguous as to whether the reinforcer is contrived or natural. Are grades in school a natural reinforcer or a contrived reinforcer? It depends on whether one’s grades are considered to be a typical aspect of the learning environment, at least within the school system, or as a contrived aspect. In any event, as with intrinsic versus extrinsic motivation, the important thing is to be able to distinguish those situations in which the reinforcers are clearly contrived—as often occurs in a behavior modification program—from those in which the reinforcers are considerably more natural. 1. A(n) 2. 3. 4. 5. reinforcer is a reinforcer that typically occurs for reinforcer is one that that behavior in that setting; a(n) typically does not occur for that behavior in that setting. You flip the switch and the light comes on. The light coming on is an example of a(n) (contrived/natural) reinforcer for flipping the switch. You thank your roommate for helping out with the housework in an attempt to motivate them to help more often. To the extent that this works, the thankyou is an example of a (contrived/natural) reinforcer; it is also an example of an (intrinsic/extrinsic) reinforcer. In applied behavior analysis, although one might initially use (contrived/ natural) consequences to first develop a behavior, the hope is that the behavior by the n will become tr associated with that behavior. c In most cases, the most important consequence for developing a highly effective slapshot in hockey will be the (contrived/natural) consequence of where the puck goes and how fast it travels. 6.4 Positive Reinforcement: Further Distinctions Shaping Positive reinforcement is clearly a great way to strengthen a behavior, but what if the behavior that you wish to reinforce never occurs? For example, what if you want to reinforce a rat’s behavior of pressing a lever but are unable to do so because the rat never presses the lever? What can you do? The solution is to use a procedure called shaping. Shaping is the gradual creation of new behavior through reinforcement of successively closer approximations to that behavior. With our rat, we could begin by delivering food whenever it stands near the lever. As a result, it begins standing near the lever more often. We then deliver food only when it is facing the lever, at which point it starts facing the lever more often. In!a similar manner, step-by-step, we reinforce touching the lever, then placing a paw on the lever, and then pressing down on the lever. When the rat finally presses down on the lever with enough force, it closes the microswitch that activates the food magazine. The rat has now earned a reinforcer on its own. After a few more experiences like this, the rat begins to reliably press the lever to earn food. By reinforcing successively closer approximations to the target behavior, we have managed to teach the rat an entirely new behavior. Another example of shaping: How do you teach a dog to catch a Frisbee? Many people simply throw the Frisbee at the dog, at which point the dog probably wonders what on earth has gotten into its owner as the Frisbee sails over its head. Or possibly the dog runs after the Frisbee, picks it up after it falls on the ground, and then makes the owner chase after it to get the Frisbee back. Karen Pryor (1999), a professional animal trainer, recommends the following procedure. First, reinforce the dog’s behavior of taking the Frisbee from your hand and immediately returning it. Next, raise the criterion by holding the Frisbee in the air to make the dog jump for it. When this is well established, toss the Frisbee slightly so the dog jumps and catches it in midair. Then toss it a couple of feet so the dog has to run after it to catch it. Now gradually throw it farther and farther so the dog has to run farther and farther to get it. Remember to provide lots of praise each time the dog catches the Frisbee and returns it. Shaping is obviously a fundamental procedure for teaching animals to perform tricks. During such training, the trainers often use a sound, such as a click from a handheld clicker, to reinforce the behavior. The sound has been repeatedly paired with food so that it has become a secondary reinforcer. The benefit of using a sound as a reinforcer is that it can!be presented immediately upon the occurrence of the behavior, even if the animal is some distance away. Also, if food were presented each time the correct behavior occurred, the animal would quickly satiate, at which point the food would become ineffective as a reinforcer. By using a secondary reinforcer such as a click, with food delivered only intermittently, satiation will take longer to occur, thereby allowing for longer training sessions. 243 Chapter 6 Operant Conditioning: Introduction Joseph Sohm/Visions of America/Corbis 244 An excellent demonstration of the power of shaping. 6.4 Positive Reinforcement: Further Distinctions 245 Advice for the Lovelorn Dear Dr. Dee, My partner has a terrible tendency to boss me around. I have tried to counter this tendency by being especially nice to him, to show him how I would like to be treated, but the problem seems to be getting worse. He also refuses to discuss it or see a counselor. He says I’m too sensitive and that I’m looking for problems where none exist. What should I do? Just About Hadenough Dear Just, You should first recognize that some people have a long history of reinforcement for being dominant or aggressive, and that it is sometimes difficult to alter such tendencies. In fact, you might eventually have to bail out of this relationship, particularly because he refuses to discuss what seems to be an obvious problem. Nevertheless, you might also wish to consider the possibility that you are inadvertently reinforcing his aggressiveness. Remember how, in the opening vignette to this chapter, the young woman reacted to her partner’s angry demands by a show of affection. While this might have reduced his anger in the short run, it might also have reinforced his tendency to be aggressive. After all, not only was his anger effective in getting her to hurry up, it also resulted in a hug. The next time he wants her to hurry up or desires affection, what better way than to get angry? (Just to be clear, however, this does not mean that his bad behavior is her fault. It’s simply important to note that we can sometimes reinforce bad behavior accidentally, and if we are aware of such contingencies, we can then change them in a manner that will be beneficial for both parties.) As a first step, you might wish to take note of the situations in which your partner becomes bossy. If it appears that you might be inadvertently reinforcing his bossiness by being nice to him when he acts that way, you could try to instead provide little or no attention when he’s bossy and lots of attention when he behaves appropriately. Can this work? In her book Don’t Shoot the Dog, Karen Pryor (1999) relates the following story about a woman who implemented just such a program: A young woman married a man who turned out to be very bossy and demanding. Worse yet, his father, who lived with them, was equally given to ordering his daughter-in-law about. It was the girl’s mother who told me this story. On her first visit she was horrified at what her daughter was going through. “Don’t worry, Mother,” the daughter said. “Wait and see.” The daughter formed the practice of responding minimally to commands and harsh remarks, while reinforcing with approval and affection any tendency by either man to be pleasant and thoughtful. In a year, she had turned them into decent human beings. Now they greet her with smiles when she comes home and leap up—both of them—to help with the groceries. (p. 30) By reinforcing successively closer approximations toward decent behavior and not reinforcing bossy behavior (yet still responding minimally to their requests), this woman was apparently able to shape more appropriate behavior in her husband and father-in-law. Remember, though, that such problems are often difficult to manage and may require professional help. Or you might simply have to abandon the relationship. Behaviorally yours, 246 Chapter 6 Operant Conditioning: Introduction Most of our behaviors have, to some extent, been learned or modified through shaping. For example, when children first learn to eat with a knife and fork, parents might praise even very poor attempts. Over time, though, they expect better and better performance before offering praise. In a similar manner, we gradually shape the child’s behavior of dressing appropriately, speaking politely, and writing legibly. And shaping is not confined merely to childhood. All of us are in the position of receiving constant feedback about our performance—be it driving, cooking a meal, or slam-dunking a basketball—thus allowing us to continually modify our behaviors and improve our skills. In such circumstances, it is usually the natural consequences of the behavior—the extent to which we are successful or unsuccessful— that provide the necessary reinforcement for gradual modifications of the behavior. And Furthermore Training Ishmael Although the principles of reinforcement and shaping are easy enough to understand, applying these principles is another matter. In this case, there is no substitute for the experience of shaping behavior in a live animal. Unfortunately, many students live in apartments or dormitories where trainable pets, such as dogs and cats, are not allowed. Fortunately, apartment dwellers are often allowed to keep fish, and some fish are in fact quite trainable. A quick search of the Internet, for example, will reveal several examples of goldfish that have been trained to swim through hoops, push balls around, and even ring a bell for food. To illustrate the process of training a fish, let us consider some training that Russ conducted with Ishmael, a two-inch long, dark blue, male Betta splendens (Siamese fighting fish). The interesting thing about training a male Betta is that two types of reinforcers are available. One is food; however, Bettas are sometimes fussy eaters (unlike goldfish) and should be given only a few bites of food per day, which can significantly shorten training sessions. The other type of reinforcer is the presentation of a mirror. The mirror image that the fish sees of itself is perceived as another male, which then elicits the fixed action pattern of an aggressive threat display. Bettas being rather feisty in nature, the opportunity to aggress like this can be used as a positive reinforcer to strengthen some other behavior (Melvin, 1985; T. Thompson, 1963). As a demonstration of the effectiveness of using the mirror to train Ishmael, mirror presentations were made contingent upon the behavior of turning a half circle. During an initial baseline period, Ishmael’s clockwise and counterclockwise circling tendencies were recorded throughout a 10-minute session with 10 mirror presentations presented noncontingently (independent of any behavior) at random points in time. (Question: Why include mirror presentations in the baseline period?) During this period, Ishmael showed a slight preference for turning in a counterclockwise direction (refer to Figure 6.4). A clockwise turn was therefore selected for initial training. Mirror presentations were initially made contingent upon successively closer approximations to the required behavior (i.e., slight turns in the correct direction were initially reinforced, with subsequent reinforcement requiring progressively more complete turns). Shaping proceeded 6.4 Positive Reinforcement: Further Distinctions 247 Figure 6.4 Number of clockwise and counterclockwise turns made by Ishmael across different phases of the demonstration. Baseline 30 Clockwise reinforced Counterclockwise reinforced Clockwise reinforced Number of turns 25 Clockwise Counterclockwise 20 15 10 5 0 0 2 4 6 8 10 12 14 16 18 20 Session rapidly, with Ishmael quickly establishing a pattern of clockwise turns. In session 9, the requirement was reversed, with counterclockwise turns being reinforced and clockwise turns being extinguished. In this phase, even with shaping, counterclockwise turns did not become well established until session 13. A reversal was then attempted in which clockwise turns were again reinforced and counterclockwise turns were extinguished (hence, overall, this was an ABCB design). Unfortunately, although Ishmael’s behavior seemed to be changing in the intended direction, in session 20, the number of turns in either direction—as well as, it seemed, his general activity level—dropped sharply. During attempted sessions on subsequent days, he mostly sat at the bottom of the tank and reacted only minimally to the mirror. It appeared as though long-term habituation had set in such that the mirror was no longer sufficiently reinforcing to motivate the target behavior. Training was therefore terminated. Nevertheless, despite this less-than-ideal conclusion, the results obtained do suggest that mirror presentation was, throughout much of the study, functioning as an effective reinforcer for Ishmael’s circling behavior. If you are interested in training your own Betta, there is plenty of information available on the Web, including videos; just search for “Betta training” or “Betta tricks.” As for the question about including mirror presentations in the baseline period, mirror presentation by itself generates a lot of excitement and movement in a Betta. Hence, noncontingent presentation of the mirror during the baseline period controls for the increase in circling that might occur simply due to the increased movement caused by mirror presentation alone. For further information on shaping, you might wish to visit Karen Pryor’s Web site on “clicker training” (shaping through the use of clicks as secondary reinforcers); a link is provided at the end of the chapter under ‘Further Exploration’. Clicker training has become very popular with dog owners as 248 Chapter 6 Operant Conditioning: Introduction Quick Quiz M well as zookeepers and is being used to shape behavior in everything from birds to horses and even llamas and elephants. (Refer also to “Training Ishmael” in the And Furthermore box.) 1. Shaping is the creation of new operant behavior through the reinforcement of s a to that behavior. 2. In clicker training with dogs, the click is a s reinforcer , which that has been established by first pairing it with f reinforcer. is a p 3. The advantages of using the click as a reinforcer is that it can be delivered . The use of a clicker can also help prevent the animal i on the reinforcer. from becoming s In Review To create a summary of this chapter, we recommend one paragraph to identify the components of an operant contingency and describe how the consequences for a behavior affect the future probability of the behavior. Be sure to include the symbols used to represent the components of a contingency. You will need another paragraph to describe the four types of contingencies, including the meaning of “positive” and “negative” in this context. A final paragraph could describe the role of shaping as well as intrinsic/extrinsic and natural/contrived reinforcers. For this chapter, we strongly recommend that you try out the procedures that you’ve just studied. If you have a pet, you can use the links that follow to get ideas for tricks you could train. Also pay attention to the ways in which we reinforce the behaviors of our family or friends, such by smiling or praising them for behaviors we enjoy. What are some of the ways in which people reinforce others through text messages, or in social media? Further Exploration For more information about clicker training and its application to a wide!variety of training situations, please go to Karen Pryor’s Web site: www.clickertraining.com/. There are numerous examples of shaping with the use of a clicker on YouTube. Here’s a good one that involves training a previously abused mule to gradually enter a stall that it is frightened of: https://www.youtube.com/watch?v=HCtrtbdXkVw. And if you would like to see videos of how operant conditioning can be used!to train adorable mice to do some rather complex tricks, you might!enjoy the Mouse Agility YouTube channel: www.youtube.com/channel/UCeFTkz2pCD0fhbmqQk0Timg. Your text authors don’t just write about shaping, they use it in real life! You’ve read about how Russ trained his Betta, now watch Lynne shape her cat’s behavior: https://youtu.be/dxD5iDTFvTA. Concept Review Study Questions 1. State Thorndike’s law of effect. What is operant conditioning (as defined by Skinner), and how does this definition differ from Thorndike’s law of effect? 2. Define the terms reinforcer and punisher. How do those terms differ from the terms reinforcement and punishment? 3. What is a discriminative stimulus? How does it differ from a conditioned stimulus? Define the three-term contingency and diagram an example. Be sure to include the appropriate symbols for each component. 4. Define positive reinforcement and negative reinforcement and diagram an example of each. Be sure to include the appropriate symbols for each component. 5. Define positive punishment and negative punishment and diagram an example of each. Be sure to include the appropriate symbols for each component. 6. How is negative reinforcement similar to positive punishment? Why are the two terms so often confused with each other? 7. How does immediacy affect the strength of a reinforcer? How does this often lead to difficulties for students in their academic studies? 8. Distinguish between primary reinforcers, secondary reinforcers, and generalized reinforcers and give an example of each. Why are generalized reinforcers often quite powerful? 9. Define intrinsic and extrinsic reinforcement and give an example of each. Under what conditions might extrinsic reinforcement undermine intrinsic interest? Under what conditions might it enhance intrinsic interest? 10. Define natural and contrived reinforcers and give an example of each. What type of reinforcer is commonly used in the initial stages of behavior modification, and what is the hoped-for result in the long run? 11. Define shaping and give an example. What are two advantages of using a secondary reinforcer, such as the sound of a click, as an aid to shaping? Concept Review avoidance behavior. Behavior that occurs before the aversive stimulus is presented and thereby prevents its delivery. contrived reinforcers. Reinforcers that have been deliberately arranged to modify a behavior; they are not a typical consequence of the behavior in that setting. Also called artificial reinforcers. discriminative stimulus (SD). A stimulus in the presence of which responses are reinforced and in the absence of which they are not reinforced; that is, a stimulus that signals the availability of reinforcement. discriminative stimulus for extinction (S!). A stimulus that signals the absence of reinforcement. discriminative stimulus for punishment (SDp). A stimulus that signals that a response will be punished. escape behavior. A behavior that results in the termination of an aversive stimulus. 249 250 Chapter 6 Operant Conditioning: Introduction extinction. The weakening of a behavior through the nonreinforcement of a previously reinforced behavior. extrinsic reinforcement. The reinforcement provided by a consequence that is external to, or additional to, the behavior. Engaging in the behavior does not itself function as a reinforcer for the behavior. generalized reinforcer. A type of secondary reinforcer that has been associated with several other reinforcers. Also called a generalized secondary reinforcer. intrinsic reinforcement. Reinforcement provided by the mere act of performing the behavior; engaging in the behavior is itself a reinforcer. law of effect. As originally stated by Thorndike, the proposition that behaviors that lead to a satisfying state of affairs are strengthened, or “stamped in,” while behaviors that lead to an unsatisfying or annoying state of affairs are weakened, or “stamped out.” natural reinforcers. Reinforcers that are naturally provided for a certain behavior; they are a typical consequence of the behavior within that setting. negative punishment. The removal of a stimulus (one that is usually considered pleasant or rewarding) following a response, which then leads to a decrease in the future strength of that response. negative reinforcement. The removal of a stimulus (one that is usually considered unpleasant or aversive) following a response, which then leads to an increase in the future strength of that response. operant behavior. A class of emitted responses that result in certain consequences; these consequences, in turn, affect the future frequency (or probability or strength) of those responses. operant conditioning. A type of learning in which the future frequency (or probability or strength) of a behavior is affected by its consequences. positive punishment. The presentation of a stimulus (one that is usually considered unpleasant or aversive) following a response, which then leads to a decrease in the future strength of that response. positive reinforcement. The presentation of a stimulus (one that is usually considered pleasant or rewarding) following a response, which then leads to an increase in the future strength of that response. primary reinforcer. An event that is innately reinforcing. No learning is required for it to function as a reinforcer. Also called an unconditioned reinforcer. punisher. An event that (1) follows a behavior and (2) decreases the future probability of that behavior. reinforcer. An event that (1) follows a behavior and (2) increases the future probability of that behavior. secondary reinforcer. An event that is reinforcing because it has been associated with some other reinforcer. It becomes a reinforcer as a function of learning. Also called a conditioned reinforcer. shaping. The gradual creation of new behavior through reinforcement of successively closer approximations to that behavior. three-term contingency. The relationship between a discriminative stimulus, an operant response, and a consequence (reinforcer or punisher). Chapter Test Chapter Test 31. Shaping is: (A) the reinforcement of a new operant behavior; (B) the gradual reinforcement of a new operant behavior; (C) the reinforcement of successively closer approximations to a new operant behavior; (D) the creation of new operant behavior through successive approximations to reinforcement. 20. A positive reinforcer is a stimulus, (A) the presentation of which increases the future strength of a response; (B) the presentation of which follows a response and increases the future strength of that response; (C) the presentation of which decreases the future strength of a response; (D) the presentation of which follows a response and decreases the future strength of that response. 2. Elicited behaviors are controlled by the events that (precede/follow) their occurrence, while operant behaviors are controlled by the events that (precede/follow) their occurrence. 14. An easy way to remember the three-term contingency is that you something, something, and something. 25. Behaviors that are performed for their own sake are said to be motivated; behaviors that are performed to achieve some additional incentive are said to be motivated. 11. Formally speaking, reinforcers and punishers are defined entirely by their on behavior. 8. An event is a punisher if it a behavior and the future probability of that behavior . 23. Money and praise are common examples of reinforcers. 12. If the rat does not press the lever, then it does not receive a shock. As a result, the rat no longer presses the lever. With respect to the behavior of pressing the lever, this is best considered as an example of (A) negative reinforcement, (B) negative punishment, (C) positive reinforcement, or (D) positive punishment. (Think carefully about this one.) 28. At the zoo one day, you notice a zookeeper leading a rhinoceros into a pen by repeatedly whistling at it. It is probably the case that the whistle has been paired with and is now functioning as a . 1. Compared to elicited behaviors, operant behaviors usually seem (more/ less) automatic and reflexive. 15. The three-term contingency can be thought of as an ABC sequence, in which A stands for , B stands for , and C stands for . 27. The gradual development of new operant behavior through reinforcement of successively closer (instances/approximations) of that behavior is called . 6. Operant behaviors are sometimes simply called ____________________. These can be contrasted with elicited behaviors, which Skinner called behaviors or simply . 251 252 Chapter 6 Operant Conditioning: Introduction 21. Each time a student studies at home, they are praised by their parents. As a result, they no longer study at home. This is an example of (positive/ negative) (reinforcement/punishment). 17. When combined with the words reinforcement or punishment, the word negative indicates that the consequence consists of something being (removed/unpleasant), and the word positive indicates that the consequence consists of something being (presented/pleasant). 10. The terms reinforcer or punisher refer to the specific that follows a behavior, whereas the terms reinforcement or punishment refer to the or whereby the probability of a behavior is altered by its consequences. 24. Harpreet very much enjoys hard work and often volunteers for projects that are quite demanding. For Harpreet, it is likely the case that the act of expending a lot of effort has often been and now functions as a reinforcer. 3. According to Thorndike’s , behaviors that lead to a(n) state of affairs are strengthened, whereas behaviors that lead to a(n) state of affairs are weakened. 30. A generalized secondary reinforcer is one that has become a reinforcer because it has been associated with: (A) a primary reinforcer; (B) a secondary reinforcer; (C) several secondary reinforcers; (D) several primary reinforcers; (E) several reinforcers (either primary or secondary). 19. When Beth tried to pull the tail of her dog, he bared his teeth and growled threateningly. Beth quickly pulled her hand back. The dog growled even more threateningly the next time Beth reached for his tail, and she again pulled her hand away. Eventually Beth gave up, and no longer tries to pull the dog’s tail. The dog’s behavior of baring his teeth and growling served to (positively/negatively) (punish/reinforce) Beth’s behavior of trying to pull his tail. And Beth’s behavior of pulling her hand away served to the dog’s future behavior of growling. 32. Achieving a record number of strikeouts in a game would be a (natural/ contrived) reinforcer for pitching well; receiving a bonus for throwing that many strikeouts would be a(n) reinforcer. 5. Operant behaviors are usually defined as a(n) of responses, all of which are capable of producing a certain . 16. A stimulus that signals that a response will be punished is called a for punishment. It is sometimes given the symbol . 22. Events that are innately reinforcing are called reinforcers; events that become reinforcers through experience are called reinforcers. 9. A reinforcer is usually given the symbol , while a punisher is usually given the symbol . The operant response is given the symbol .A discriminative stimulus that signals the availability of reinforcement is given the symbol . Answers to Chapter Test 26. Steven has fond memories of his mother reading fairy tales to him when he was a child. As a result, he now enjoys reading fairy tales as an adult. For Steven, the act of reading fairy tales is functioning as what type of reinforcer? (A) primary; (B) secondary; (C) intrinsic; (D) extrinsic; (E) both secondary and intrinsic. 4. Classically conditioned behaviors are said to be e by the st ; operant behaviors are said to be e by the org . 18. Referring to this chapter’s opening vignette, among the four types of contingencies described in this chapter, the woman’s response to her boyfriend most likely resulted in of his behavior, which means that his abusive behavior toward her will likely (increase/ decrease) in the future as a result of her actions. 33. Saying that a certain consequence has increased the future probability of a behavior is, within this text, considered equivalent to saying that the consequence has increased the future (strength/frequency/either) of the behavior. 7. An event is a reinforcer if it a behavior and the (present/future) probability of that behavior. 29. Major advantages of using the sound of a click for shaping are that the click can be delivered and the animal is unlikely to upon it. 13. A discriminative stimulus is a stimulus that signals that a(n) is available. It is said to “ ” for the behavior. Answers to Chapter Test 1. less 2. precede; follow 3. law of effect; satisfying; unsatisfying (or annoying) 4. elicited; stimulus; emitted; organism 5. class; consequence 6. operants; respondent; respondents 7. follows; increases; future 8. follows; decreases 9. SR; SP; R; SD 10. consequence (or event or stimulus); process; procedure 11. effect 12. D (because “lever press→ shock” is the effective contingency) 13. reinforcer; set the occasion 14. notice; do; get 15. antecedent; behavior; consequence 16. discriminative stimulus; SDp 17. removed; presented 18. positive reinforcement; increase 19. positively; punish; negatively reinforce 20. B 21. positive punishment 22. primary (or unconditioned); secondary (or conditioned) 23. generalized (or generalized secondary) 24. positively reinforced; secondary (or conditioned) 25. intrinsically; extrinsically 26. E 27. approximations; shaping 28. food; secondary reinforcer 29. immediately; satiate 30. E 31. C 32. natural; contrived 33. either 253 Chapter 7 Schedules and Theories of!Reinforcement “I don’t understand why Alvin is so distant,” Mandy commented. “He was great when we first started going out. Now it’s like pulling teeth to get him to pay attention to me.” “So why do you put up with it?” her sister asked. “I guess I’m in love with him. Why else would I be so persistent?” Chapter Outline 7.1 Schedules of Reinforcement Continuous Versus Intermittent Schedules Four Basic Intermittent Schedules Other Simple Schedules of Reinforcement Complex Schedules of Reinforcement 7.2 Theories of Reinforcement Drive Reduction Theory The Premack Principle Response Deprivation Hypothesis Behavioral Bliss Point Approach Learning Objectives 7-1 Define schedule of reinforcement. 7-2 Distinguish between continuous and intermittent schedules of reinforcement. 7-3 Identify and describe patterns of responding associated with the four main intermittent schedules of reinforcement. 7-4 Identify and describe patterns of responding associated with duration schedules and response-rate schedules of reinforcement. 7-5 Distinguish between simple and complex schedules. 7-6 Describe procedures and outcomes associated with establishing behavioral chains, including goal gradient effects. 7-7 Compare and contrast various theories of reinforcement. 254 7.1 Schedules of Reinforcement 7.1 Schedules of Reinforcement In this section, we discuss schedules of reinforcement. A schedule of reinforcement is the response requirement that must be met to obtain reinforcement. In other words, a schedule indicates what exactly has to be done for the reinforcer to be delivered. For example, does each lever press by the rat result in a food pellet, or are several lever presses required? Did your parents give you a cookie each time you asked for one, or only some of the time? And just how persistent does Mandy have to be before Alvin will pay attention to her? As you will discover in this section, different response requirements can have dramatically different effects on behavior. Many of these effects (known as schedule effects) were first observed in experiments with pigeons (Ferster & Skinner, 1957), but they also help explain some puzzling aspects of human behavior that are often attributed to internal traits or desires. Continuous Versus Intermittent Schedules A continuous reinforcement schedule is one in which each specified response is reinforced. For example, each time a rat presses the lever, it obtains a food pellet; each time the dog rolls over on command, it gets a treat; and each time Karen turns the ignition in her car, the motor starts. Continuous reinforcement (CRF) is very useful when a behavior is first being shaped or strengthened. For example, when using a shaping procedure to train a rat to press a lever, reinforcement should be delivered for each approximation to the target behavior. Similarly, if we wish to encourage a child to always brush their teeth before bed, we would do well to initially praise them each time they do so. An intermittent (or partial) reinforcement schedule is one in which only some responses are reinforced. For example, perhaps only some of the rat’s lever presses result in a food pellet, and perhaps your mother gave you a cookie only occasionally when you asked for one. Intermittent reinforcement obviously characterizes much of everyday life. Not all concerts we attend are enjoyable, not every person we invite out on a date accepts, and not every date that we go out on leads to an enjoyable evening. And although we might initially praise a child each time they properly complete their homework, we might soon praise them only occasionally in the belief that such behavior should persist in the absence of praise. There are four basic (or simple) types of intermittent schedules: fixed ratio, variable ratio, fixed interval, and variable interval. We will describe each one along with the characteristic response pattern it produces. Note that this characteristic response pattern is the stable pattern that emerges once the organism has had considerable exposure to the schedule. Such stable patterns are known as steady-state behaviors, in contrast to the more variable patterns of behavior that are evident when an organism is first learning a schedule. 255 Quick Quiz A 256 Chapter 7 Schedules and Theories of Reinforcement 1. A s 2. 3. 4. 5. of reinforcement is the r requirement that must be met in order to obtain reinforcement. On a c reinforcement schedule (abbreviated ), reinforcement each response is reinforced, whereas on an i schedule, only some responses are reinforced. The latter is also called a reinforcement schedule. p Each time you flick the light switch, the light comes on. The behavior of flicking schedule of reinforcement. the light switch is on a(n) When the weather is very cold, you are sometimes unable to start your car. The behavior of starting your car in very cold weather is on a(n) schedule of reinforcement. S e are the different effects on behavior produced by different response requirements. These are the stable patterns of behavior that emerge once the organism has had sufficient exposure to the schedule. -st Such stable patterns are known as st behaviors. Four Basic Intermittent Schedules Fixed Ratio Schedules On a fixed ratio (FR) schedule, reinforcement is contingent upon a fixed, predictable number of responses. For example, on a fixed ratio 5 schedule (abbreviated FR 5), a rat has to press the lever five times to obtain a food pellet. On an FR 50 schedule, it has to press the lever 50 times to obtain a food pellet. Similarly, earning a dollar for every 10 carburetors assembled on an assembly line is an example of an FR 10 schedule, while earning a dollar for each carburetor assembled is an example of an FR 1 schedule. Note that an FR 1 schedule is the same as a CRF (continuous reinforcement) schedule in which each response is reinforced (thus, such a schedule can be correctly labeled as either an FR 1 or a CRF). If you wish to diagram a three-term contingency that includes the schedule of reinforcement, one way to do so is to insert the schedule value in superscript following the operant response. For example, for a rat that is pressing a lever to earn food on an FR 10 schedule of reinforcement, with a light serving as the discriminative stimulus, the contingency could be diagrammed as follows: Light (SD): Lever pressFR 10 (R) → Food (SR) FR schedules generally produce a high rate of response along with a short pause following the attainment of each reinforcer (refer to Figure 7.1). This short pause is known as a post-reinforcement pause. For example, a rat on an FR 25 schedule will rapidly emit 25 lever presses, munch down the food pellet it receives, and snoop around the chamber for a few seconds before rapidly emitting another 25 lever presses. In other words, it will take a short break following each reinforcer, just as you might take a short break after reading each chapter in a textbook or completing a particular assignment. Note, too, that each pause is usually followed by a relatively quick return to a high rate of response. Thus, the typical FR pattern is described as a “break-and-run” pattern—a short break followed by a steady run of responses. 7.1 Schedules of Reinforcement Response patterns for fixed ratio (FR), variable ratio (VR), fixed interval (Fl), and variable interval (VI) schedules. This figure shows the characteristic pattern of responding on each of the four basic schedules. Notice the high response rate on the FR and VR schedules, moderate response rate on the VI schedule, and scalloped response pattern on the FI schedule. Also, both the FR and FI schedules are accompanied by post-reinforcement pauses. (Source: Modified from Nairne, 2000.) Figure 7.1 Fixed ratio schedule Variable ratio schedule Rapid responding Cumulative responses High, steady rate Postreinforcement pause Fixed interval schedule Long pause after reinforcement yields “scalloping” effect No pauses Variable interval schedule Moderate, steady rate No pauses Time Time In general, higher ratio requirements produce longer post-reinforcement pauses. This means that you will probably take a longer break after completing a long assignment than after completing a short one. Similarly, a rat will show longer pauses on an FR 100 schedule than on an FR 30 schedule. With very low ratios, such as CRF or FR 2, there may be little or no pausing other than the time it takes for the rat to munch down the food pellet. In such cases, the next reinforcer is so close—only a few lever presses away—that the rat is tempted to immediately go back to work. (If only the reinforcers for studying were so immediate!) Schedules in which the reinforcer is easily obtained are said to be very dense or rich, while schedules in which the reinforcer is difficult to obtain are said to be very lean. Thus, an FR 5 schedule is considered a very dense schedule of reinforcement compared to an FR 100. During a one-hour session, a rat can earn many more food pellets on an FR 5 schedule than 257 258 Chapter 7 Schedules and Theories of Reinforcement Quick Quiz B it can on an FR 100. Likewise, an assembly line worker who earns a dollar for each carburetor assembled (a CRF schedule) is able to earn considerably more during an eight-hour shift than is a worker who earns a dollar for every 10 carburetors assembled (an FR 10 schedule). In general, “stretching the ratio”—moving from a low ratio requirement (a dense schedule) to a high ratio requirement (a lean schedule)—should be done gradually. For example, once lever pressing is well established on a CRF schedule, the requirement can be gradually increased to FR 2, FR 5, FR 10, and so on. If the requirement is increased too quickly—for example, CRF to FR 2 and then a sudden jump to FR 20—the rat’s behavior may become erratic and even die out altogether. Likewise, if you try to raise the requirement too high—say, to FR 2000—there may be a similar breakdown in the rat’s behavior. Such breakdowns in behavior are technically known as ratio strain, which is a disruption in responding due to an overly demanding response requirement. Ratio strain is what most people would refer to as “burnout,” and it can be a big problem for students faced with a heavy workload. Some students, especially those who have a history of getting by with minimal work, may find it increasingly difficult to study under such circumstances and may even choose to drop out of college. If they had instead experienced a gradual increase in workload over a period of several months or years, they might have been better able to put forth the needed effort to succeed. 1. On a(n) 2. 3. 4. 5. 6. 7. 8. 9. 10. schedule, reinforcement is contingent upon a fixed number of responses. A schedule in which 15 responses are required for each reinforcer is abbreviated . A mother finds that she always has to make the same request three times before her child complies. The mother’s behavior of making requests is on a(n) schedule of reinforcement. An FR 1 schedule of reinforcement can also be called a schedule. A fixed ratio schedule tends to produce a (high/low) rate of response, along p . with a p An FR 200 schedule of reinforcement will result in a (longer/shorter) pause than an FR 50 schedule. The typical FR pattern is sometimes called a b pattern, with a pause that and-r is followed immediately by a (high/low) rate of response. An FR 12 schedule of reinforcement is (denser/leaner) than an FR 75 schedule. A dense schedule of reinforcement can also be referred to as a r schedule of reinforcement. Over a period of a few months, Aaron changed from complying with each of his father’s requests to complying with every other request, then with every third request, and so on. The father’s behavior of making requests has the been subjected to a procedure known as “s .” r 7.1 Schedules of Reinforcement 259 11. Graduate students often have to complete an enormous amount of work in the initial year of their program. For some students, the workload is far beyond anything they have previously encountered. As a result, their study behavior may become increasingly (erratic/stereotyped) throughout the year, a process s . known as r Study Tip: Post-reinforcement pauses may have important implications for studying. On the one hand, they suggest that there may be a natural tendency for us to take a break after completing a certain amount of work. On the other hand, when students take a break from studying, they often have difficulty starting up again. A possible explanation is that students, unlike pigeons and rats who have little to do during a post-reinforcement pause, tend to engage in highly reinforcing activities during a study break, such as playing computer games or watching television. A possible solution to this problem might be to ensure that study breaks, especially the ones you intend to be brief, do not involve anything particularly interesting; for example, rather than watching television, you might instead do some light housework, a bit of exercise, or perhaps just sit and listen to some quiet music. Like a pigeon with a break-and-run pattern, you may now find it much easier to resume studying. As for more reinforcing activities, such as TV watching, these can be saved for the end of the study session, where you can partake of them as major reinforcers for a job well done (though, as you!will learn in Chapter 10, there are questions concerning the extent to which such “self-reinforcement” constitutes real reinforcement). Another useful trick, which can be used whenever you have difficulty sitting down to study, is the just-get-started (JGS) tactic (e.g., Pychyl, 2010). In this case, you set yourself a very small and easily attained goal, such as to study for at least 5 or 10 minutes with the option to quit if you still “don’t feel like studying.” Many students find that once the allotted time has passed, it is surprisingly easy to carry on—in the same way that once a pigeon starts responding on an FR schedule, it has a strong tendency to complete the entire run of responses up to the next reinforcer. Many students report finding the JGS tactic to be helpful in reducing the tendency to procrastinate (such as when working on term papers), and you might wish to experiment with it yourself. Variable Ratio Schedules On a variable ratio (VR) schedule, reinforcement is contingent upon a varying, unpredictable number of responses. For example, on a variable ratio 5 (VR 5) schedule, a rat has to emit an average of five lever presses for each food pellet, with the number of lever responses on any particular trial varying between, say, 1 and 10. Thus, the number of required lever presses might be three for the first pellet, six for the second pellet, one for the third pellet, seven for the fourth pellet, and so on, with the overall average being five lever presses for each reinforcer. Similarly, on a VR 50 schedule, the number of required lever presses may vary between 1 and 100, with the average being 50. VR schedules generally produce a high and steady rate of response, often with little or no post-reinforcement pause (refer to Figure 7.1). A post-reinforcement pause is especially unlikely to occur when the minimum response requirement in 260 Chapter 7 Schedules and Theories of Reinforcement the schedule is very low, such as a VR 50 schedule in which the requirement varies between 1 and 100 as opposed to a VR 50 in which the requirement varies between, say, 10 and 90 (Schlinger, Derenne, & Baron, 2008). This is understandable if you consider that each response on the former schedule has the potential of resulting in a reinforcer even if a reinforcer has just been obtained. The real world is filled with examples of VR schedules. Some predatory behaviors, such as that shown by cheetahs, have a strong VR component in that only some attempts at chasing down prey are successful. In humans, only some acts of politeness receive an acknowledgment, only some residents who are called upon by canvassers will make a contribution, and only some TV shows are enjoyable. Many sports activities, such as shooting baskets in basketball and shots on goal in hockey, are also reinforced largely on a VR schedule. As I am writing this passage, a colleague stopped by and joked that his golf drive is probably on a VR 200 schedule. In other words, he figures that an average of about 1 in every 200 drives is a good one. I (Russ Powell) replied that my own drives are on a much leaner schedule, with the result that ratio strain has set in, which is fancy behaviorist talk for “I so rarely hit the ball straight that I have just about given up playing.” Variable ratio schedules help account for the persistence with which some people display certain maladaptive behaviors. Gambling is a prime example in this regard: the unpredictable nature of these activities results in a very high rate of behavior. In fact, the behavior of a gambler playing a slot machine is the classic example of a VR schedule in humans. Certain forms of aberrant social behavior may also be accounted for by VR schedules. For example, why do some men persist in using flippant or crass remarks to introduce themselves to women, when the vast majority of women view such remarks negatively? One reason is that a small minority of women actually respond favorably, thereby intermittently reinforcing the use of such remarks. For example, Kleinke, Meeker, and Staneske (1986) found that although 84% of women surveyed rated the opening line “I’m easy. Are you?” as poor to terrible, 14% rated it as either very good or excellent! This equates to around a VR 7 schedule of reinforcement, which is certainly rich enough to encourage the behavior. Variable ratio schedules of reinforcement may also facilitate the development of an abusive or exploitative relationship. At the start of a relationship, the individuals involved typically provide each other with an enormous amount of positive reinforcement (a very dense schedule). This strengthens the relationship and increases each partner’s attraction to the other. As the relationship progresses, such reinforcement naturally becomes somewhat more intermittent. In some situations, however, this process becomes malignant, with one person (let us call this person the Taker) providing reinforcement on an extremely intermittent basis, and the other person (the Giver) working incredibly hard to obtain that reinforcement. Because the process evolves gradually (a process of slowly “stretching the ratio”), the Giver may have little awareness of what is happening until the 7.1 Schedules of Reinforcement 261 abusive pattern is well established. What would motivate such an unbalanced process? One source of motivation is that the less often the Taker reinforces the Giver, the more attention (reinforcement) they receive from the Giver. In other words, the Giver works so hard to get the Taker’s attention that they actually reinforce the very process of being largely ignored by that partner. Of course, it does not necessarily have to be a one-way process, and there may be relationships in which the partners alternate the role of Giver and Taker. The result may be a volatile relationship that both partners find exciting but that is constantly on the verge of collapse due to frequent periods in which each partner experiences “ratio strain.” , of responses. un 2. A variable ratio schedule typically produces a (high/low) rate of behavior (with/ without) a post-reinforcement pause. 3. An average of 1 in 10 people approached by a panhandler actually gives them (be precise money. The behavior of panhandling is on a(n) and use the abbreviation) schedule of reinforcement. 4. As with an FR schedule, an extremely lean VR schedule can result in r . s Fixed Interval Schedules On a fixed interval (FI) schedule, reinforcement is contingent upon the first response after a fixed, predictable period of time. For a rat on a fixed interval 30-second (FI 30-sec or FI 30”) schedule, the first lever press after a 30-second interval has elapsed results in a food pellet. Following that, another 30 seconds must elapse before a lever press will again produce a food pellet. Any lever pressing that occurs during the interval, before the 30-second period has elapsed, is ineffective. Similarly, trying to phone a business that opens in exactly 30 minutes will be effective only after the 30!minutes have elapsed, with any phone calls before that being ineffective. FI schedules often produce a “scalloped” (upwardly curved) pattern of responding, consisting of a post-reinforcement pause followed by a gradually increasing rate of response as the interval draws to a close (refer to Figure 7.1). For example, a rat on an FI 30-sec schedule will likely emit no lever presses at the start of the 30-second interval. This will be followed by a few tentative lever presses perhaps midway through the interval, with a gradually increasing rate of response thereafter. By the time the interval draws to a close and the reinforcer is imminent, the rat will be emitting a high rate of response, with the result that the reinforcer will be attained as soon as it becomes available. Would the behavior of trying to phone a business that opens in 30 minutes also follow a scalloped pattern? If we have a watch available, it probably would not. We would simply look at our watch to determine when the 30!minutes have elapsed and then make the phone call. The indicated time!would be a discriminative stimulus (SD) for when the reinforcer is available (i.e., the!business is open), and we would wait until the appropriate time before phoning. But what about the behavior of looking at your watch during Quick Quiz C 1. On a variable ratio schedule, reinforcement is contingent upon a 262 Chapter 7 Schedules and Theories of Reinforcement Quick Quiz D the 30 minutes (the reinforcer for which would be noticing that the interval has elapsed)? You are unlikely to spend much time looking at your watch at the start of the interval. As time progresses, however, you will begin looking at it more and more frequently. In other words, your behavior will follow the typical scalloped pattern of responding. The distribution of study sessions throughout the semester can also show characteristics of an FI scallop, which can again contribute to students’ tendency to procrastinate in their studying. At the start of a course, many students engage in little or no studying, given that the first exam is some distance away. This is followed by a gradual increase in studying as the first exam approaches. The completion of the exam is again followed by little or no studying until the next exam approaches. Unfortunately, these postreinforcement pauses are often too long, with the result that many students obtain much poorer marks than they would have if they had studied at a steadier pace throughout. (Note, however, that studying for exams is not a pure example of an FI schedule, because a certain amount of work must be accomplished during the interval to obtain the reinforcer of a good mark. On a pure FI schedule, any responding that happens during the interval is essentially irrelevant.) 1. On a fixed interval schedule, reinforcement is contingent upon the 2. 3. 4. 5. , pr period of response following a(n) ____________________. If I have just missed the bus when I get to the bus stop, I know that I have to wait 15 minutes for the next one to come along. Given that it is absolutely freezing out, I snuggle into my parka as best I can and grimly wait out the interval. Every once in a while, though, I emerge from my cocoon to take a quick glance down the street to see if the bus is coming. My behavior of looking for the bus is on (use the abbreviation) schedule of reinforcement. a(n) In the example in question 2, I will probably engage in (few/frequent) glances at the start of the interval, followed by a gradually (increasing/decreasing) rate of glancing as time passes. Responding on an FI schedule is often characterized by a sc p pattern of responding consisting of a p followed by a gradually (increasing/decreasing) rate of responding as the interval draws to a close. On a pure FI schedule, any response that occurs (during/following) the interval is irrelevant. Variable Interval Schedules On a variable interval (VI) schedule, reinforcement is contingent upon the first response after a varying, unpredictable period of time. For a rat on a variable interval 30-second (VI 30-sec or VI 30”) schedule, the first lever press after an average interval of 30 seconds will result in a food pellet, with the actual interval on any particular trial varying between, say, 1 and 60 seconds. Thus, the number of seconds that must pass before a lever press will produce a food pellet could be 8 seconds for the first food pellet, 55 seconds for the second pellet, 24 seconds for the third, and so 7.1 Schedules of Reinforcement 263 on, the average of which is 30 seconds. Similarly, if each day you are waiting for a bus and have no idea when it will arrive, then looking down the street will be reinforced by the appearance of the bus after a varying, unpredictable period of time—for example, 2 minutes the first day, 12 minutes the next day, 9 minutes the third day, and so on, with an average interval of, say, 10 minutes (VI 10-min). VI schedules usually produce a moderate, steady rate of response, often with little or no post-reinforcement pause (refer to Figure 7.1). By responding at a relatively steady rate throughout the interval, the rat on a VI 30-sec schedule will attain the reinforcer almost as soon as it becomes available. The behavior of checking your cell phone for notifications is maintained on a VI schedule. Our phones provide all sorts of reinforcers, from simple entertainment to meaningful communication with other people, and those little notification alerts pop up every so often. Even if you’ve just checked your phone, to see if any texts have arrived or if someone has liked the photo you posted, you may feel compelled to check it again. Because VI schedules produce steady, predictable response rates, they are often used to investigate other aspects of operant conditioning, such as those involving choice between alternative sources of reinforcement. You will encounter examples of this when we discuss choice behavior in Chapter 10. response following a __________________, un___________________ period of ___________________. 2. You find that by frequently switching stations on your radio, you are able to hear your favorite song an average of once every 20 minutes. Your behavior of switching stations is thus being reinforced on a schedule. 3. In general, variable interval schedules produce a (low/moderate/high) and (steady/fluctuating) rate of response with little or no ____________________. Comparing the Four Basic Schedules The four basic schedules produce quite different patterns of behavior, which vary in both the rate of response!and in the presence or absence of a post-reinforcement pause. These characteristics are summarized in Table 7.1. As can be seen, ratio schedules (FR and VR) produce higher rates of response than do interval schedules (FI and VI). This makes sense because the reinforcer in ratio schedules is entirely “response contingent”; that is, it depends entirely on the number of responses emitted. A rat on a VR 100 schedule can double the number of food pellets earned in a one-hour session by doubling its rate of lever pressing. Similarly, someone who works in direct sales can double the number of sales they make during a day by doubling the number of customers they call or e-mail (assuming they continue to give an adequate sales pitch to each customer). Compare this to an interval schedule in which reinforcement is mostly “time contingent.” For example, on an FI 1-minute schedule, a rat Quick Quiz E 1. On a variable interval schedule, reinforcement is contingent upon the ________________ Chapter 7 Schedules and Theories of Reinforcement Table 7.1 Characteristic response rates and post-reinforcement pauses for each of the four basic intermittent schedules. These are only general characteristics; they are not found under all circumstances. For example, an FR schedule with a very low response requirement, such as FR 2, is unlikely to produce a post-reinforcement pause. By contrast, an FR schedule with a very high response requirement, such as FR 2000, might result in ratio strain and a complete cessation of responding. FR VR FI VI Response rate High High Increasing Moderate Post-reinforcement pause Yes No Yes No can earn no more than 50 reinforcers in a 50-minute session regardless of how rapidly it presses the lever. Under such circumstances, responding at a high rate throughout each interval does not pay off and is essentially a waste of energy. Instead, it makes more sense to respond in a way that will simply maximize the possibility of attaining the reinforcer soon after it becomes available. On an FI schedule, this means responding at a gradually increasing rate as the interval draws to a close; on a VI schedule, this means responding at a moderate, steady pace throughout the interval. Note also that fixed schedules (FR and FI) tend to produce postreinforcement pauses, whereas variable schedules (VR and VI) do not. On a variable schedule, there is often the possibility of a relatively immediate reinforcer, even if one has just attained a reinforcer, which tempts one to immediately resume responding. By comparison, on a fixed schedule, attaining one reinforcer means that the next reinforcer is necessarily some distance away. On an FR schedule, this results in a short post-reinforcement pause before grinding out another set of responses; on an FI schedule, this results in a somewhat longer pause followed by a gradually increasing rate of response as the interval draws to a close and the reinforcer becomes imminent. Dilbert © 1993 Scott Adams. Used by permission of!Universal Uclick. All rights reserved 264 Which of these workers is on a ratio schedule of reinforcement? 7.1 Schedules of Reinforcement is because the reinforcer in such schedules is entirely r contingent, meaning that the rapidity with which responses are emitted (does/ does not) affect how soon the reinforcer is obtained. 2. On schedules, the reinforcer is largely time contingent, meaning that the rapidity with which responses are emitted has (little/ considerable) effect on how soon the reinforcer is obtained. 3. In general, (variable/fixed) schedules produce little or no post-reinforcement pausing because such schedules often provide the possibility of relatively reinforcement, even if one has just obtained a i reinforcer. 4. In general, schedules produce post-reinforcement pauses because obtaining one reinforcer means that the next reinforcer is necessarily quite (distant/near). Other Simple Schedules of Reinforcement Duration Schedules On a duration schedule, reinforcement is contingent on performing a behavior continuously throughout a period of time. On a fixed duration (FD) schedule, the behavior must be performed continuously for a fixed, predictable period of time. For example, the rat must run in the wheel for 60 seconds to earn one pellet of food (an FD 60-sec schedule). Likewise, Julie may decide that her son can watch television each evening only after he completes two hours of studying (an FD 2-hr schedule). On a variable duration (VD) schedule, the behavior must be performed continuously for a varying, unpredictable period of time. For example, the rat must run in the wheel for an average of 60 seconds to earn one pellet of food, with the required time varying between 1 second and 120 seconds on any particular trial (a VD 60-sec schedule). And Julie may decide to reinforce her son’s studying with cookies and other treats at varying points in time that happen to average out to about one treat every 30 minutes (a VD 30-min schedule). (Question: How do FD and VD schedules differ from FI and VI schedules?) Although duration schedules are sometimes useful in modifying certain human behaviors, such as studying, they are in some ways rather imprecise compared to the four basic schedules discussed earlier. With FR schedules, for example, one knows precisely what was done to achieve the reinforcer, namely, a certain number of responses. On an FD schedule, however, what constitutes “continuous performance of behavior” during the interval could vary widely. With respect to wheel running, for example, a “lazy” rat could dawdle along at barely a walk, while an “energetic” rat might rotate the wheel at a tremendous pace. Both would receive the reinforcer. Similarly, Julie’s son might read only a few pages during his two-hour study session or charge through several chapters; in either case, he would receive the reinforcer of being allowed to watch television. Remember too, from Chapter 6, how reinforcing the mere performance of an activity with no regard to level of performance might undermine a person’s intrinsic interest in that activity. This danger obviously Quick Quiz F 1. In general, (ratio/interval) schedules tend to produce a high rate of response. This 265 266 Chapter 7 Schedules and Theories of Reinforcement applies to duration schedules; one therefore needs to be cautious in their use. Nevertheless, many successful writers write according to an FD schedule of reinforcement (i.e., they write for a fixed amount of time each day), although there are also some who write according to an FR schedule—that is, they strive to complete a certain number of words or pages each day (Currey, 2013; Silvia, 2018). An informal survey of your fellow students will likely show that they too vary in the extent to which they study for a certain amount of time or to accomplish a certain amount of work (e.g., to finish a chapter), something you may wish to explore in optimizing your own study behavior. If you have difficulty studying for lengthy periods of time, you might especially wish to consider the use of the highly popular Pomodoro technique, which is described in the Appendix: A Brief Guide to Behavior Self-Management. Response-Rate Schedules As we have seen, different types of intermittent schedules produce different rates of response (i.e., they have different schedule effects). These different rates are essentially by-products of the schedule. However, in a response-rate schedule, reinforcement is directly contingent upon the organism’s rate of response. Let’s examine three types of responserate schedules. In differential reinforcement of high rates (DRH), reinforcement is contingent upon emitting at least a certain number of responses in a certain period of time—or, more generally, reinforcement is provided for responding at a fast rate. The term differential reinforcement means simply that one type of response is reinforced while another is not. In a DRH schedule, reinforcement is provided for a high rate of response and not for a low rate. For example, a rat might receive a food pellet only if it emits at least 30 lever presses within a period of a minute. Similarly, a worker on an assembly line may be told that they can keep their job only if they assemble a minimum of 20 carburetors per hour. By requiring so many responses in a short period of time, DRH schedules ensure a high rate of responding. Athletic events such as running, swimming, and rowing are prime examples of DRH schedules in that winning is directly contingent on a rapid series of responses. In differential reinforcement of low rates (DRL), a minimum amount of time must pass between each response before the reinforcer will be delivered—or, more generally, reinforcement is provided for responding at a slow rate. For example, a rat might receive a food pellet only if it waits at least 10 seconds between lever presses. So how is this different from an FI 10-sec schedule? Remember that on an FI schedule, responses that occur during the interval have no effect; on a DRL schedule, however, responses that occur during the interval do have an effect—an adverse effect in that they prevent reinforcement from occurring. In other words, responding during the interval must not occur in order for a response following the interval to produce a reinforcer. Human examples of DRL schedules consist of situations in which a person is required to perform an action slowly or infrequently. For example, a parent might praise a child for brushing their teeth slowly or completing homework slowly, given that going too fast generally results in sloppy performance. Once 7.1 Schedules of Reinforcement 267 the quality of performance improves, reinforcement can then be made contingent on responding at a normal speed. Some behaviors are naturally maintained on a DRL schedule. If you bake cookies for your friends every day, you may get little or no praise for your baking (once your friends have habituated to them!). If you only bake cookies once per week, you will likely get praise each time. In differential reinforcement of paced responding (DRP), reinforcement is contingent upon emitting a series of responses at a set rate—or, more generally, reinforcement is provided for responding neither too fast nor too slow. For example, a rat might receive a food pellet if it emits 10 consecutive responses, with each response separated by an interval of no less than 1.5 and no more than 2.5 seconds. Similarly, musical activities, such as playing in a band or dancing to music, require that the relevant actions be performed at a specific pace. People who are very good at this are said to have a good sense of timing or rhythm. Further examples of DRP schedules can be found in noncompetitive swimming or running. People often perform these activities at a pace that is fast enough to ensure benefits to health and a feeling of well-being, yet not so fast as to result in exhaustion and possible injury. In fact, even competitive swimmers and runners, especially those who compete over long distances, will often set a specific pace throughout much of the race. Doing so ensures that they have sufficient energy at the end for a last-minute sprint (DRH) to the finish line, thereby maximizing their chances of clocking a good time. 2. 3. 4. 5. for a varying period of time; on an (FI/FD) schedule, reinforcement is contingent upon the first response after a fixed period of time. As Tessa sits quietly in the doctor’s office, her father occasionally gives her a hug as a reward. As a result, Tessa is more likely to sit quietly on future visits to the doctor. This is an example of a(n) schedule of reinforcement. In practicing the slow-motion exercise known as tai chi, Tung noticed that the more slowly he moved, the more thoroughly his muscles relaxed. This is an reinforcement of example of d behavior (abbreviated ). In a video game, the faster you destroy all the targets, the more bonus points you obtain at the end of the game. This is an example of behavior reinforcement of ). (abbreviated Frank discovers that he feels better if he jogs at a nice, even rhythm that is neither reinforcement too fast nor too slow. This is an example of behavior (abbreviated ). of Noncontingent Schedules On a noncontingent schedule of reinforcement, the reinforcer is delivered independently of any response. In other words, a response is not required for the reinforcer to be obtained. Such schedules are also called response-independent schedules. There are two types of noncontingent schedules: fixed time and variable time. Quick Quiz G 1. On a (VD/VI) schedule, reinforcement is contingent upon responding continuously 268 Chapter 7 Schedules and Theories of Reinforcement Quick Quiz H On a fixed time (FT) schedule, the reinforcer is delivered following a fixed, predictable period of time, regardless of the organism’s behavior. For example, on a fixed time 30-second (FT 30-sec) schedule, a pigeon receives access to food every 30 seconds regardless of its behavior. Likewise, many people receive Christmas gifts each year, independently of whether they have been naughty or nice—an FT 1-year schedule. FT schedules therefore involve the delivery of a “free” reinforcer following a predictable period of time. On a variable time (VT) schedule, the reinforcer is delivered following a varying, unpredictable period of time, regardless of the organism’s behavior. For example, on a variable time 30-second (VT 30-sec) schedule, a pigeon receives access to food after an average interval of 30 seconds, with the actual interval on any particular trial ranging from, say, 1 second to 60 seconds. Similarly, you may coincidentally run into an old friend about every three months on average (a VT 3-month schedule). VT schedules therefore involve the delivery of a free reinforcer following an unpredictable period of time. (Question: How do FT and VT schedules differ from FI and VI schedules?) 1. On a non schedule of reinforcement, a response is not required to obtain a reinforcer. Such a schedule is also called a r____________________ -i____________________ schedule of reinforcement. 2. Every morning at 7:00 A.M., a robin perches outside Marilyn’s bedroom window and begins singing. Given that Marilyn very much enjoys the robin’s singing, this 24-hour is an example of a schedule of reinforcement (abbreviated ____________________). 3. For farmers, rainfall is an example of a noncontingent reinforcer that is typically schedule delivered on a (abbreviated ____________________). Noncontingent reinforcement may account for some forms of superstitious behavior. In the first investigation of this possibility, Skinner (1948b) presented pigeons with food every 15 seconds (FT 15-sec) regardless of their behavior. Although you might think that such free reinforcers would have little effect on the pigeons’ behavior (other than encouraging them to stay close to the feeder), quite the opposite occurred. Six of the eight pigeons began to display ritualistic patterns of behavior. For example, one bird began turning counterclockwise circles, while another repeatedly thrust its head into an upper corner of the chamber. Two other pigeons displayed a swaying pendulum motion of the head and body. Skinner believed these behaviors developed because they had been accidentally reinforced by the coincidental presentation of food. For example, if a pigeon just happened to turn a counterclockwise circle before food delivery, that behavior would be accidentally reinforced and increase in frequency. This would increase the likelihood of the same behavior occurring the next time food was delivered, which would further strengthen it. The eventual result would be a wellestablished pattern of turning circles, as though turning circles somehow caused the food to appear. 7.1 Schedules of Reinforcement Some researchers have argued that Skinner’s evidence for superstitious behavior in the pigeon may not be as clear-cut as he believed. They claim that at least some of the ritualistic behaviors he observed may have consisted of innate tendencies, almost like fidgeting behaviors, that are often elicited during a period of waiting (Staddon & Simmelhag, 1971). These tendencies, which are discussed in Chapter 12, are known as adjunctive behaviors. Nevertheless, other experiments have replicated the effect of noncontingent reinforcement on the development of superstitious behavior. Ono (1987), for example, placed students in a booth that contained three levers and a counter. The students were told, “if you do something, you may get points on the counter” (p. 263). They were also told to get as many points as possible. In reality, the points were delivered on either an FT or VT schedule, so the students’ behavior actually had no effect on point delivery. Nevertheless, most students developed at least temporary patterns of superstitious lever pulling; that is, they pulled the lever as though it were effective in producing points. Interestingly, one student started with lever pulling but then coincidentally received a point after simply touching the counter. This led to a superstitious pattern of climbing on the counter and touching different parts of the apparatus, apparently in the belief that this action produced the points. She jumped off the apparatus at just the time that she received another point, which led to a superstitious pattern of repeatedly jumping in the air and touching the ceiling! After several minutes of this, she finally quit, apparently as a result of fatigue. Professional athletes and gamblers are particularly prone to the development of superstitions, some of which may evolve in the manner that Skinner suggests. Under constant threat of losing their position to an eager newcomer, professional athletes are constantly on the lookout for anything that might enhance their performance. As a result, unusual events that precede a fine performance, such as humming a certain tune or wearing an unusual article of clothing, may be quickly identified and deliberately reproduced in the hopes of reproducing that performance. Gamblers display even stronger tendencies toward the development of superstitions, probably because the activity in which they are engaged is even more uncertain in its outcome. Bingo players, for example, commonly carry lucky pendants, stuffed animals, or pieces of jewelry to each game and are often adamant (almost pathologically so) about obtaining cards that contain certain patterns or are drawn from the top or bottom of the stack. Many of these rituals probably evolved because they were at one time associated with a big win. Herrnstein (1966) noted that superstitious behaviors can sometimes develop as by-products of contingent reinforcement for some other behavior. For example, a businessperson might believe it is important to impress customers with a firm handshake, when in fact it is merely the handshake, and not the firmness of the handshake, that is the critical factor. (Unfortunately, such a superstition could have serious consequences if the businessman then attempts to branch out into the Asian market, where a firm handshake is often regarded as a sign of disrespect.) Similarly, some managers might come 269 270 Chapter 7 Schedules and Theories of Reinforcement Quick Quiz I to believe that “pushing the panic button” is an effective way to deal with crises, simply because it is usually followed by a successful outcome. What they fail to realize is that a low-key approach might have been equally if not more effective—and certainly a lot less stressful. Question: Although Skinner’s (1948b) original demonstration of superstitious behavior involved the use of a fixed time schedule, is superstitious behavior in humans more likely to develop under a fixed or variable time schedule? To answer this, think about the types of situations in which you are particularly likely to find superstitious behavior in humans. Is it in situations that involve predictable events or unpredictable events? Obviously, it is unpredictable events, such as games of chance, sports, hunting and fishing (having a “lucky” hat, lure, or ritual), and so forth. In this sense, at least from a human perspective, superstitious behavior can be seen as an attempt to make an unpredictable situation more predictable. 1. When noncontingent reinforcement happens to follow a particular behavior, that behavior may (increase/decrease) in strength even though it is not required to obtain the reinforcer. Such behavior is referred to as s behavior. 2. Herrnstein (1966) noted that such behaviors can sometimes develop as a byproduct of (contingent/noncontingent) reinforcement for some other behavior. 3. As shown by the kinds of situations in which such behaviors develop in humans, these behaviors seem most likely to develop on a(n) (VT/FT) schedule of reinforcement. What happens if a noncontingent schedule of reinforcement is superimposed on a regular contingent schedule of reinforcement? What if, for example, a pigeon responding on a VI schedule of food reinforcement also receives extra reinforcers for free? Will the pigeon’s rate of response on the VI schedule increase or decrease? In fact, the pigeon’s rate of response on the VI schedule will decrease (Rachlin & Baum, 1972). Suggestive evidence of this effect can even be found among professional athletes. One!study, conducted several years ago, found that major league pitchers who had signed long-term contracts showed a significant decline in number of innings pitched relative to pitchers who only signed a one-year contract (O’Brien, Figlerski, Howard, & Caggiano, 1981). Insofar as a long-term contract or a guaranteed purse (as in boxing) virtually guarantees a hefty salary regardless of performance, these results are consistent with the possibility that athletic performance might sometimes decline when the money earned is no longer contingent on level of performance. At this point, you might be thinking that noncontingent reinforcement is all bad, given that it leads to superstitious behavior in some situations and to poor performance in others. In fact, noncontingent reinforcement is sometimes quite beneficial. More specifically, it can be an effective means for reducing the frequency of maladaptive behaviors. For example, children who act out often do so to obtain attention. If, however, they are given a sufficient 7.1 Schedules of Reinforcement 271 amount of attention on a noncontingent basis, they will no longer have to act out to obtain it. Noncontingent reinforcement has even been shown to reduce the frequency of self-injurious behavior. Such behavior, which can consist of head-banging or biting chunks of flesh out of one’s arm, is sometimes displayed by people with certain types of developmental disorders and can be notoriously difficult to treat. In some of these cases, the behavior appears to be maintained by the attention it elicits from caretakers. Research has shown, however, that if the caretakers provide the individual with plenty of attention on a noncontingent basis, the frequency of their self-injurious behavior may be greatly reduced (e.g., Hagopian, Fisher, & Legacy, 1994). In a sense, they no longer have to injure themselves to receive attention because they are now receiving lots of attention for free. The beneficial effects of noncontingent reinforcement can also be seen as providing empirical support for the value of what Carl Rogers (1959), the famous humanistic psychologist, called “unconditional positive regard.” Unconditional positive regard refers to the love, respect, and acceptance that one receives from significant others, regardless of one’s behavior. Rogers assumed that such regard is a necessary precondition for the development of a healthy personality. From a behavioral perspective, unconditional positive regard can be viewed as a form of noncontingent social reinforcement, which can indeed have beneficial effects. In fact, it seems likely that proper child rearing requires healthy doses of both noncontingent reinforcement, which gives the child a secure base from which to explore the world and take risks, and contingent reinforcement, which helps shape the child’s behavior in appropriate ways, maximize skill development, and prevent the development of passivity. Thus, Abraham Maslow (1971), another famous humanistic psychologist, argued that child rearing should be neither too restrictive nor too lenient, which in behavioral terms can be taken to imply that the social reinforcement children receive should be neither excessively contingent nor excessively noncontingent. delivering additional food on a VT 60-second schedule. As a result, the rate of response on the VR schedule is likely to (increase/decrease/remain unchanged). 2. A child who is often hugged during the course of the day (which they seem to enjoy), regardless of what they are doing, is in humanistic terms receiving unconditional positive regard. In behavioral terms, the hugs are a form of social reinforcement. As a result, this child may be non (more/less) likely to act out in order to receive attention. Complex Schedules of Reinforcement All of the schedules previously described are relatively simple in that there is only one basic requirement. On the other hand, a complex schedule consists of a combination of two or more simple schedules. There are a wide variety of such schedules, three of which are described here. Two other types of complex Quick Quiz J 1. During the time that a rat is responding for food on a VR 100 schedule, we begin 272 Chapter 7 Schedules and Theories of Reinforcement schedules—multiple schedules and concurrent schedules—are discussed in later chapters. Conjunctive Schedules A conjunctive schedule is a type of complex schedule in which the requirements of two or more simple schedules must be met before a reinforcer is delivered. For example, on a conjunctive FI 2-minute FR 100 schedule, reinforcement is contingent upon completing 100 lever presses plus at least one lever press following a two-minute interval. Many of the contingencies that we encounter in everyday life are examples of conjunctive schedules. The wages you earn on a job are contingent upon working a certain number of hours each week and doing a sufficient amount of work so that you will not be fired. Likewise, Jae’s fiancée might have chosen to marry her because she is kind and humorous and interesting and drives a Porsche. With any one of these components missing, she would have paid little attention to her. Quick Quiz K Adjusting Schedules In an adjusting schedule, the response requirement changes as a function of the organism’s performance while responding for the previous reinforcer. For example, on an FR 100 schedule, if the rat completes all 100 responses within a five-minute interval, we then increase the requirement to 110 responses (FR 110) within a five-minute period. In other words, because the rat has performed so well, we expect even better performance in the future. In a similar fashion, when Tara displayed excellent ability in mastering her violin lessons, she and her parents decided to increase the amount she had to learn each week. And when Lily’s high school students performed poorly on their exams, she gradually decreased the amount of material they had to learn each week. (This sort of change could be beneficial if the original amount students had to learn was unreasonable and they are now much better able to learn a smaller amount of material. Conversely, it could be a problem if it results in the class no longer covering the necessary content and students not gaining the necessary skills. In that case, the problem may reside in how the students are being taught or in their level of preparedness for the class.) The process of shaping involves an adjusting schedule insofar as the criterion for reinforcement is raised depending on the animal’s performance. As soon as the rat has learned to stand near the lever to get food, the criterion is raised to touching the lever, placing a paw on the lever, and so forth. The requirement for reinforcement changes as soon as the rat has successfully met the previous requirement. 1. A(n) schedule is one that consists of a combination of two or more simple schedules that must be completed (in any order/in a particular order) before the reinforcer is delivered. 2. In a(n) schedule, the response requirement changes as a function of how the organism performed while responding for the previous reinforcer. 7.1 Schedules of Reinforcement 273 3. To the extent that a gymnast is trying to improve their performance, their behavior is likely on a(n) ____________________ schedule of reinforcement; to the extent that their performance is judged according to both the form and quickness of schedule of their moves, their behavior is on a(n) reinforcement. Chained Schedules A chained schedule consists of a sequence of two or more simple schedules, each of which has its own SD and the last of which results in a terminal reinforcer. In other words, the person or animal must work through a series of component schedules to obtain the sought-after reinforcer. A chained schedule differs from a conjunctive schedule in that the two component schedules must be completed in a particular order, which is not required in a conjunctive schedule. As an example of a chained schedule, a pigeon in a standard operant conditioning chamber is presented with a VR 20 schedule on a green key, followed by an FI 60-sec schedule on a red key, which leads to the terminal reinforcer of food. Thus, an average of 20 responses on the green key will result in a change in key color to red, following which the first response on the red key after a 10-second interval will be reinforced by food. The food is the terminal reinforcer that supports the entire chain. This chain can be diagrammed as follows: Green key (SD): PeckVR 20 (R) → Red key (SR/SD): PeckFI 60” (R) → Food (SR) Note that the presentation of the red key is both a secondary reinforcer for completing the preceding VR 20 schedule and an SD for responding on the subsequent FI 60-sec schedule. Note, too, that this is an example of a two-link chain, with the VR 20 schedule constituting the first, or initial, link and the FI 60-sec schedule constituting the second, or terminal, link. By adding yet another schedule to the start of the chain, we can create a three-link chain, which we can diagram as follows (excluding the abbreviations to save space): White key: PeckVI 30” → Green key: PeckVR 20 → Red key: PeckFI 60” → Food In this case, both the green and red keys function as secondary reinforcers for responding in the previous link to help maintain responding throughout the chain. of which has its own ____________________ and the last of which results in a t____________________ r____________________. 2. Within a chain, completion of each of the early links ends in a (primary/secondary) reinforcer, which also functions as the ____________________ for the next link of the chain. Once pigeons learn which schedule is associated with which key, they generally show the appropriate response patterns for those schedules. In the preceding example, this would be a moderate, steady rate of response on the Quick Quiz L 1. A chained schedule consists of a sequence of two or more simple schedules, each 274 Chapter 7 Schedules and Theories of Reinforcement white key, a high rate of response on the green key, and a scalloped pattern of responding on the red key. Nevertheless, responding tends to be somewhat weaker in the earlier links of a chain than in the later links. This can be seen most clearly when each link consists of the same schedule. For example, Kelleher and Fry (1962) presented pigeons with a three-link chained schedule with each link consisting of an FI 60-sec schedule: White key: PeckFI 60” → Green key: PeckFI 60” → Red key: PeckFI 60” → Food The pigeons displayed very long pauses and a slow rate of response on the white key compared to the other two keys. The greatest amount of responding occurred on the red key. Why would the earlier links of the chain be associated with weaker responding? One way of looking at it is that in the later links, the terminal reinforcer is more immediate and hence more influential, while in the early links, the terminal reinforcer is more distant and hence less influential (remember that delayed reinforcement is less effective than immediate reinforcement). Another way of looking at it is that the secondary reinforcers supporting behavior in the early links are less directly associated with food and are therefore relatively weak (e.g., the green key is associated with food only indirectly through its association with the red key). From this perspective, a chained schedule can be seen as the operant equivalent of higher-order classical conditioning—in which, for example, a tone (CS 1) associated with food (US) elicits less salivation than the food does, and a light (CS 2) associated with the tone elicits less salivation than the tone does. Similarly, in the example of the chained schedule, the red key associated with the food is a less powerful reinforcer than the food, and the green key associated with the red key is a less powerful reinforcer than the red key. (If you find that you can no longer remember the concept of higher-order classical conditioning, you should go back and review it.) The difference in response strength between the early and later links in a chain is representative of a more general behavioral principle known as the goal gradient effect. The goal gradient effect is an increase in the strength and/or efficiency of responding as one draws near to the goal. For example, rats running through a maze to obtain food tend to run faster and make fewer wrong turns as they near the goal box (Hull, 1932). Similarly, a student writing an essay is likely to take shorter breaks and work more intensely as they near the end. Dolphin trainers are well aware of the goal gradient effect. Dolphins who are trained to perform long chains of behaviors have a tendency to drift toward “sloppy” performance during the early parts of the chain, and trainers have to be vigilant to ensure that the dolphin’s behavior is not reinforced when this occurs (Pryor, 1975). (Perhaps the most profound example of a goal gradient, however, is that shown by people who desperately need to urinate and move very quickly as they near the washroom.) An efficient way to establish responding on a chained schedule is to train the final link first and the initial link last, a process known as backward chaining. Using the pigeon example, the pigeon would first be trained to 7.1 Schedules of Reinforcement respond on the red key to obtain food. This will establish the red key as a secondary reinforcer through its association with food. The presentation of the red key can then be used to reinforce responding on the green key. Once this is established, the presentation of the green key can be used to reinforce responding on the white key. In the preceding examples, each link in the chain required the same type of behavior, namely, key pecking. It is also possible to create behavior chains in which each link consists of a different behavior. For example, a rat might have to climb a ladder and then pull a rope to earn food. This can be diagrammed, with abbreviations, as follows: Ladder (SD): Climb ladder (R) → Rope (SR/SD): Pull rope (R)→ Food (SR) Note that the sight of the rope is both a secondary reinforcer for climbing the ladder and a discriminative stimulus for pulling the rope. As with the previous examples, backward chaining would be the best way to train this sequence of behaviors. Thus, the rat would first be trained to pull the rope for food. Once this is established, it would be taught to climb the ladder to get to the rope, with the sight of the rope acting as a secondary reinforcer for climbing the ladder. In this manner, step-by-step, very long chains of behavior can be established. In one reported example, a rat was trained to go up a ladder, cross a platform, climb a rope, cross a bridge, get into a little elevator box, release the pulley holding the box, lower the box “paw over paw” to the floor, and press a button to obtain the food (Pryor, 1975). Of course, each of these behaviors also had to be shaped (through reinforcement of successive approximations to the target behavior). Shaping and chaining are thus the basic means by which circus and marine animals are trained to perform some remarkable feats (refer to Figure 7.2). Most human endeavors involve response chains, some of which are very long. The act of reading this chapter, for example, consists of reading section after section, until the terminal reinforcer of completing the entire chapter Keystone/Hulton Archive/Getty Images Ross Anania/Media Bakery Figure 7.2 Through shaping and chaining, animals can be taught to display some surprising behaviors. 275 276 Chapter 7 Schedules and Theories of Reinforcement Quick Quiz M has been attained. Completing each section serves as both a secondary reinforcer for having read that section as well as an SD for reading the next section. Reading the chapter is in turn part of a much larger chain of behaviors that includes attending lectures, taking notes, and studying, the terminal reinforcer for which is passing the course. Fortunately, backward chaining is not required for the development of such chains, because language enables us to describe to one another the required sequence of behaviors (such as by providing a course syllabus). In other words, for humans, complex response chains are often established through the use of instructions. Unfortunately, in the case of very long chains, such as completing a course, the terminal reinforcer is often extremely distant, with the result that behavior is easily disrupted during the early part of the chain (remember the goal gradient principle). This is yet another reason for why it is much easier to be a diligent student the night before the midterm than during the first week of the semester. Can anything be done to alleviate this problem? One possibility is to make the completion of each link in the chain more salient (i.e., more noticeable), thereby enhancing its value as a secondary reinforcer. Novelists, for example, need to write hundreds, or even thousands, of pages before the terminal reinforcer of a completed book is attained. To keep themselves on track, some novelists keep detailed records of their progress, such as charting the number of words written each day as well as the exact dates on which chapters were started and completed (Wallace & Pear, 1977). These records outline their achievements, thereby providing a much-needed source of secondary reinforcement throughout the process. Similarly, students sometimes keep detailed records of the number of hours studied or pages read. They might also compile a “to-do” list of assignments and then cross off each item as it is completed. Crossing off an item provides a clear record that a task has been accomplished and also functions as a secondary reinforcer that helps motivate us (Lakein, 1973). In fact, it can be so reinforcing that many people will actually add a task to a to-do list after they have completed it, simply to have the pleasure of crossing it off! 1. Responding tends to be weaker in the (earlier/later) links of a chain. This is an example of the g____________________ g____________________ effect, in which the strength or efficiency of responding (increases/decreases) as the organism approaches the goal. 2. An efficient way to train a complex chain, especially in animals, is through b____________________ chaining, in which the (first/last) link of the chain is trained first. However, this type of procedure usually is not required with verbally proficient humans, with whom behavior chains can be quickly established through the use of ins____________________. 3. One suggestion for enhancing our behavior in the early part of a long response chain is to make the completion of each link more (salient/gradual), thereby enhancing its value as a s____________________ reinforcer. 7.2 Theories of Reinforcement 7.2 Theories of Reinforcement In this section, we briefly discuss some major theories of reinforcement. We begin with Clark Hull’s early drive reduction view of reinforcement. This is followed by a brief description of a highly influential approach known as the Premack principle. This principle is of immense practical importance and has helped revolutionize the manner in which the process of reinforcement is now conceptualized. In fact, two other theoretical approaches that we discuss—the response deprivation hypothesis and the bliss point approach— can be viewed as outgrowths of the Premack principle. Drive Reduction Theory An early approach to understanding reinforcement, and one that was strongly championed by Hull (1943), is drive reduction theory. In drive reduction theory, an event is reinforcing to the extent that it is associated with a reduction in some type of physiological drive. Thus, food deprivation produces a “hunger drive,” which propels the animal to seek out food. When food is obtained, the hunger drive is reduced. At the same time, the behavior that preceded this drive reduction, and led to the food, is automatically strengthened. In very simple terms (in actuality, the theory is more complex than this), if a hungry rat in a maze turns left just before it finds food in the goal box, the act of turning left in the maze will be automatically strengthened by the subsequent reduction in hunger. We touched upon this theory in Chapter 6 when we noted that primary reinforcers are often those events that seem to reduce a physiological need. From this perspective, secondary reinforcers are events that have become reinforcers because they have been associated, either directly or indirectly, with a primary reinforcer and, hence, with some type of drive reduction. Thus, a person may enjoy collecting cookbooks because cooking is associated with eating food, which in turn is associated with a reduction in hunger. According to Hull, all reinforcers are associated, either directly or indirectly, with some type of drive reduction. In Chapter 6, we noted that a major problem with this physiological view of reinforcement is that some reinforcers do not seem to be associated with drive reduction. A rat will press a lever to obtain access to a running wheel, a chimpanzee will press a button so that it can obtain a peek into another room, and teenagers will spend considerable amounts of money to be exposed to earsplitting levels of music. It is difficult to see how such events are associated with a reduction in some type of physiological need. Instead, it seems as though the motivation for such behavior exists more in the nature of the reinforcing stimulus than in the state of the organism. Incentive motivation is motivation that is derived from some property of the reinforcer, as opposed to an internal drive state. Playing a video game for the fun of it, attending a concert because you enjoy the music, and working to earn enough money to buy a Porsche are examples of behaviors that are 277 278 Chapter 7 Schedules and Theories of Reinforcement Quick Quiz N motivated by incentives. Even events that seem to be clearly associated with drive reduction can be strongly affected by incentive factors. For example, going to a restaurant for a meal might be largely driven by hunger; however, the fact that you prefer a restaurant that serves hot, spicy food is an example of incentive motivation. The spiciness of the food plays no role in the reduction of hunger; it is simply a form of sensory stimulation that you also find highly reinforcing. In conclusion, most theorists no longer believe that drive reduction theory can offer a comprehensive account of reinforcement. Some recent approaches instead emphasize observable behavior patterns as opposed to hypothetical internal processes in their explanation of reinforcement. A major step in this direction was the Premack principle. 1. According to drive reduction theory, an event is reinforcing if it is associated with a reduction in some type of p____________________ drive. reinforcer is one that has been associated, directly or indirectly, with a p____________________ reinforcer that reduces a drive. 3. A major problem with drive reduction theory is that ___________________________ _________________________________________________________________. 4. The motivation that is derived from some property of the reinforcer is called ____________________ motivation. 5. Research has shown that hungry rats will perform more effectively in a T-maze when the reinforcer for a correct response (right turn versus left turn) consists of several small pellets as opposed to one large pellet (Capaldi, Miller, & Alptekin, 1989). The fact that several small bites of food are a more effective reinforcer than one large bite is consistent with the notion of (drive reduction/incentive motivation). 2. According to this theory, a s The Premack Principle Remember how Skinner defined reinforcers (and punishers) by their effect on behavior? This unfortunately presents us with a problem. In the real world, it would be nice to know ahead of time whether a certain event can function as a reinforcer. One way to do this, of course, would be to take something the person or animal seems to like and use that as a reinforcer. But it is not always easy to determine what a person or animal likes. Moreover, events that we believe should be liked might not actually function as reinforcers. To a five-year-old boy, a kiss from his mother is great if he needs comforting, but not when he is trying to show off to his friends. Fortunately, the Premack principle provides a more objective way to determine whether something can be used as a reinforcer (Premack, 1965). The Premack principle is based on the notion that reinforcers can often be viewed as behaviors rather than stimuli. For example, rather than saying that lever pressing was reinforced by food (a stimulus), we could say that lever pressing was reinforced by the act of eating food (a behavior). Similarly, rather than saying that playing appropriately was reinforced by television, we could instead say 7.2 Theories of Reinforcement that it was reinforced by watching television. When we view reinforcers in this manner—as behaviors rather than stimuli—then the process of reinforcement can be conceptualized as a sequence of two behaviors: (1) the behavior that is being reinforced, followed by (2) the behavior that is the reinforcer. Moreover, by comparing the probability or frequency of two behaviors, we can determine whether one can be used as a reinforcer for the other. More specifically, the Premack principle states that a high-probability (or high-frequency) behavior can be used to reinforce a low-probability (or lowfrequency) behavior. For example, when a rat is hungry, eating food has a higher likelihood of occurrence than running in a wheel. This means that eating food, which is the high-probability behavior (HPB), can be used to reinforce the target behavior of running in a wheel, which is the lowprobability behavior (LPB). In other words, there is a very good chance that the rat will run in the wheel to obtain access to the food. This can be!diagrammed as follows (omitting the SD and using square brackets to indicate the relative probabilities): Target behavior Consequence Running in wheel [LPB] (R) → Eating food [HPB] (SR) On the other hand, if the rat is not hungry, eating food is less likely to occur than running in a wheel. In this case, running in a wheel can be used as a reinforcer for the target behavior of eating food. In other words, the rat will eat food, even though it is not hungry, to obtain access to the wheel. Target behavior Consequence Eating food [LPB] (R) → Run in wheel [HPB] (SR) By focusing on the relative probabilities (or relative frequencies) of behaviors, the Premack principle allows us to quickly identify potential reinforcers. If, for example, Kailey spends only a few minutes each morning doing chores, but at least an hour reading comic books, then the opportunity to read comic books (the higher probability behavior) can be used to reinforce doing chores (the lower probability behavior). Do chores [LPB] (R) → Read comic books [HPB] (SR) In fact, if you want an easy way to remember the Premack principle, just think of Grandma’s rule: First you work (a low-probability behavior), then you play (a high-probability behavior). The Premack principle has proven to be very useful in applied settings. For example, a person with autism who spends many hours each day rocking back and forth might be very unresponsive to consequences that are normally reinforcing for others, such as receiving praise. The Premack principle, however, suggests that the opportunity to rock back and forth could possibly be used as an effective reinforcer for another behavior that we might wish to strengthen, such as interacting with others. Thus, the Premack principle is a handy principle to keep in mind when confronted by a situation in which normal reinforcers seem to have little effect. 279 280 Chapter 7 Schedules and Theories of Reinforcement Quick Quiz O The Premack principle can also be useful for students. A common recommendation is that students should reward themselves (present a reinforcer to themselves) following each successful study session. Some students, however, will complain that they just can’t think of a good reward in such situations. The Premack principle, however, provides a solution; if studying is a low-probability behavior and playing computer games is a high-probability behavior, then it should be possible to use gaming as a reward for studying. (As noted earlier, however, there are!certain difficulties with the concept of “self-reinforcement” which will be discussed in Chapter 10.) 1. The Premack principle proposes that reinforcers can often be viewed as 2. 3. 4. 5. 6. 7. rather than stimuli. For example, rather than saying that the rat’s lever pressing was reinforced with food, we could say that it was food. It also distinguishes between reinforced with behaviors in terms of their relative (probability/frequency/either term can be used). The Premack principle states that a(n) _________________-p_________________ behavior can be used as a reinforcer for a(n) ________________________________________________ behavior. According to the Premack principle, if you crack your knuckles three times per hour and burp 20 times per hour, the opportunity to . may be an effective reinforcer for If you drink five soda pops each day and only one glass of orange juice, the can likely be used as a reinforcer opportunity to drink . for drinking If “Chew bubble gum → Play video games” is a diagram of a reinforcement procedure based on the Premack principle, then chewing bubble gum must be a (lower/higher)-probability behavior than playing video games. Similar to the Premack principle, in Grandma’s rule, first you (work/play), which is a(n) (LPB/HPB), and then you (work/play), which is a(n) (LPB/HPB). In the Premack principle of reinforcement, the low-probability behavior (precedes/ follows) the high-probability behavior, with the former being the (target . behavior/reinforcer) and the latter being the Response Deprivation Hypothesis The Premack principle requires us to know the relative probabilities (or frequencies) of two behaviors before we can judge whether one will be an effective reinforcer for the other. But what if we have information on only one behavior? Is there any way that we can tell whether that behavior can function as a reinforcer before actually trying it out? The response deprivation hypothesis states that a behavior can serve as a reinforcer when (1) access to that behavior is restricted and (2) its frequency thereby falls, or is in danger of falling, below its preferred level of occurrence (Timberlake & Allison, 1974). The preferred level of an activity is 7.2 Theories of Reinforcement its baseline level of occurrence when the animal (or person) can freely engage in that activity. For example, imagine that a rat typically runs for one hour a day whenever it has free access to a running wheel. This one hour per day is the rat’s preferred level of running. If the rat is allowed free access to the wheel for only 15 minutes per day, it will be unable to reach this preferred level and will be in a state of deprivation with regard to running. According to the response deprivation hypothesis, the rat will now be willing to work (e.g., press a lever) to obtain additional time on the wheel. Lever press (R) → Running in a wheel (SR) The response deprivation approach also provides a general explanation for why contingencies of reinforcement are effective. Contingencies of reinforcement are effective to the extent that they create a condition in which the organism is confronted with the possibility of a certain response falling below its preferred level. Take Kailey, who enjoys reading comic books each day. If her parents demand that she complete her chores each day before being allowed to read comic books, her baseline level of free comic book reading will drop to zero. As a result, deprived of comic book reading, she will now be willing to do chores to restore her level of comic book reading back to its preferred level. Do chores (R) → Read comic books (SR) You will notice that the diagram given here is the same as that given for the Premack principle; in fact, both approaches will often lead to the same type of contingency being established. But the interpretation of why it works is different. In this case, reading comic books is a reinforcer simply because the contingency threatens to push free comic book reading to below its preferred rate of occurrence. The relative probabilities of the two behaviors are irrelevant, meaning that it does not matter if the probability of reading comic books at the outset is higher or lower than the probability of doing chores. Even if Kailey is a hard worker who spends much of her time doing chores anyway, we could get her to do even more chores by threatening to reduce her comic book time. The only thing that matters is whether comic book reading is in danger of falling below its preferred level of occurrence if the contingency is not met. Thus, the response deprivation hypothesis is applicable to a wider range of conditions than the Premack principle. (Question 4 in the following Quick Quiz may help clarify this.) To help distinguish between the Premack principle and the response deprivation hypothesis, ask yourself whether the contingency is based on the frequency of one behavior relative to a second behavior (in which case, it is an example of the Premack principle), or the frequency of a single behavior which we are being, or are in danger of being, deprived of (in which case it is an example of the response deprivation hypothesis). 281 Quick Quiz P 282 Chapter 7 Schedules and Theories of Reinforcement 1. According to the response deprivation hypothesis, a response can serve as a reinforcer if free access to the response is (provided/restricted) and its frequency ) level of occurrence. then falls (above/below) its baseline (or pr 2. If a child normally watches four hours of television per night, we can make television watching a reinforcer if we restrict free access to the television to (more/less) than four hours per night. 3. The response deprivation hypothesis differs from the Premack principle in that we need only know the baseline frequency of the (reinforced/reinforcing) behavior. 4. Kailey typically watches television for two hours per day and reads comic books for one hour per day. You set up a contingency whereby Kailey must watch 2.5!hours of television each day in order to have any access to her comic books. According to the Premack principle, this will likely be an (effective/ineffective) contingency. According to the response deprivation hypothesis, this will likely be an (effective/ineffective) contingency. Behavioral Bliss Point Approach The response deprivation hypothesis assumes there is an optimal level of behavior that an organism strives to maintain. This same assumption can be made for the manner in which an organism distributes its behavior between two or more activities. According to the behavioral bliss point approach, an organism with free access to alternative activities will distribute its behavior in such a way as to maximize overall reinforcement (Allison, 1983). For example, a rat that can freely choose between running in a wheel and exploring a maze might spend two hours per day running in a wheel and two hours exploring a maze—that is, the same amount of time exploring the maze as running. This distribution of behavior represents the optimal distribution of activities for those two activities—that is, the behavioral bliss point—for that particular rat. Note that this optimal distribution of behavior is based on the notion that each activity is freely available. When activities are not freely available—such as when the two activities are intertwined in a contingency of reinforcement— then the optimal distribution may become unattainable. What if, for example, a contingency is created in which our rat has to run in the wheel for two hours per day in order to obtain just one hour of maze exploration? It will now be impossible for the rat to reach its bliss point for these two activities. To obtain the preferred two hours of maze exploration, the rat would have to run in the wheel for four hours per day, which is far beyond its preferred level. A reasonable assumption as to what the rat will do in that circumstance is that it will compromise by distributing its activities in such a way as to draw as near as possible to its bliss point. For instance, it might choose to run a total of three hours per day to obtain 1.5 hours of maze exploration. That is not as enjoyable as the ideal distribution of two hours of running and two hours of maze exploration; however, given the contingencies, it will have to do (i.e., the rat will engage in a bit more running than it would prefer as well as a bit less maze exploration). Likewise, most of us are forced to spend several more hours working and several fewer hours enjoying the finer things in life than we would prefer if we were independently wealthy and could do whatever we wanted. The behavioral bliss point for our varied activities is essentially unattainable. Faced with certain 7.2 Theories of Reinforcement 283 contingencies that must be met in order to survive, we instead distribute our activities in such a way as to draw as near to the bliss point as possible. The behavioral bliss point approach assumes that organisms attempt to distribute their behavior so as to maximize overall reinforcement. This, of course, is a very rational way to behave. In Chapter 10, you will encounter an alternative theory, known as melioration theory, which maintains that organisms, including people, are not that rational and that various processes often entice the organism away from maximization. Note, too, that none of the theories discussed in this chapter take account of an animal’s innate tendencies toward certain patterns of behavior, which may affect how easily certain behaviors can be trained. In Chapter 12, you will encounter a theory that does take account of such tendencies. approach, an organism that (is forced to/can freely) engage in alternative activities will distribute its behavior in such a way as to (optimize/balance) the available amount of reinforcement. 2. Contingencies of reinforcement often (disrupt/enhance) the preferred distribution of behavior such that it is (easy/impossible) to obtain the optimal amount of reinforcement. 3. Given this state of affairs, the organism will redistribute its activities to get as (near to/far from) the bliss point as possible. Quick Quiz Q 1. According to the behavioral Advice for the Lovelorn Dear Dr. Dee, I recently began dating a classmate. We get along really well at school, so it seemed like we would be a perfect match. Unfortunately, once we started dating, our relationship seemed to lose a lot of its energy, and our lives seemed a lot less satisfying. Someone suggested that we must each have an unconscious fear of commitment. What do you think? Less Than Blissful Dear Less, I suppose it is possible that you have an unconscious fear of commitment—if there is such a thing as an unconscious fear of commitment. On the other hand, it may be that the amount of time you spent interacting with one another at school is actually the optimal amount of time given the various reinforcers available in your relationship. Spending additional time together (which also means spending less time on alternative activities) has, for each of you, resulted in a distribution of behavior that is further removed from your behavioral bliss point. Obviously, a good relationship should move you toward your bliss point, not away from it. Try spending less time together at school (perhaps take different classes) and see if your relationship outside of school becomes more fulfilling. Or go back to just being friends-at-school. Behaviorally yours, 284 Chapter 7 Schedules and Theories of Reinforcement In Review To summarize this chapter, you should define each described schedule of reinforcement and its associated response pattern. You could write one paragraph for the four basic intermittent schedules of reinforcement and another paragraph for duration and response-rate schedules. A third paragraph could distinguish between simple schedules and complex schedules as well as behavioral chains. A fourth paragraph could compare the various theories of reinforcement, including their strengths and weaknesses. This chapter is ideal for using your own experiences to reflect upon the material. Can you think of a time that you experienced “ratio strain”? Do you have any complex behaviors that were established through chaining procedures? Do you find yourself spending too much time on social media because of its schedule of reinforcement? Describe those patterns and identify the relevant schedules of reinforcement. Perhaps you can even think of ways to alter a schedule in order to change some behavior patterns that you don’t want! For Further Exploration If you play video games, this chapter has likely made you wonder about the schedules of reinforcement in the games you play. You might find these observations, by a behavior analyst who is also a gamer, to be very interesting: https://behavioralinquiry.com/2016/04/15/ behavioral-science-in-video-games/. Beyond video games, other aspects of screen time can have profound effects on our behavior. Read this column for a discussion about how we may spend more time scrolling than we might like: https:// www.psychologytoday.com/ca/blog/tech-happy-life/201901/ the-vegas-effect-our-screens. For a clear illustration of backward chaining, check out this dog training video. While you’re watching, think about how it could be done with a “clicker” instead of hot dogs!: https://catchdogtrainers.com/ backchaining-train-the-last-behavior-first-2019-10-16/. Study Questions 1. What is a schedule of reinforcement? Distinguish between continuous and intermittent schedules of reinforcement. 2. Name and define the four basic intermittent schedules and describe (or graph) the typical response pattern produced by each. 3. Name and define two types of duration schedules and two types of noncontingent schedules. How do they each differ from FI and VI schedules of reinforcement? Concept Review 4. Name and define three types of response-rate schedules. Give an example of each. 5. What is a conjunctive schedule, and what is an adjusting schedule? Give an example of each. In what way does shaping involve the use of an adjusting schedule? 6. What is a chained schedule? Diagram and label an example of a chained schedule. How does a chained schedule differ from a conjunctive schedule? 7. What is the best way to establish responding on a chained schedule in animals? What is the relationship between the goal gradient effect and a chained schedule of reinforcement? 8. Describe the drive reduction theory of reinforcement. What is a major difficulty with this theory? What is incentive motivation? 9. Outline the Premack principle. Outline the response deprivation hypothesis. Using the target behavior of increasing the amount you study and the consequence of ice skating, given an example of each approach that clearly differentiates between the two. 10. Describe the behavioral bliss point approach to reinforcement and the effect that a contingency of reinforcement can have on attaining one’s bliss point. Concept Review adjusting schedule. A schedule in which the response requirement changes as a function of the organism’s performance while responding for the previous reinforcer. behavioral bliss point approach. The theory that an organism with free access to alternative activities will distribute its behavior in such a way as to maximize overall reinforcement. chained schedule. A schedule consisting of a sequence of two or more simple schedules, each with its own SD and the last of which results in a terminal reinforcer. complex schedule. A schedule that consists of a combination of two or more simple schedules. conjunctive schedule. A type of complex schedule in which the requirements of two or more simple schedules must be met, in any order, before a reinforcer is delivered. continuous reinforcement schedule. A schedule in which each specified response is reinforced. differential reinforcement of high rates (DRH). A schedule in which reinforcement is contingent upon emitting at least a certain number of responses in a certain period of time—or, more generally, reinforcement is provided for responding at a fast rate. differential reinforcement of low rates (DRL). A schedule in which a minimum amount of time must pass between each response before 285 286 Chapter 7 Schedules and Theories of Reinforcement the reinforcer will be delivered—or, more generally, reinforcement is provided for responding at a slow rate. differential reinforcement of paced responding (DRP). A schedule in which reinforcement is contingent upon emitting a series of responses at a set rate—or, more generally, reinforcement is provided for responding neither too fast nor too slow. drive reduction theory. A theory that holds that an event is reinforcing to the extent that it is associated with a reduction in some type of physiological drive. fixed duration (FD) schedule. A schedule in which reinforcement is contingent upon continuous performance of a behavior for a fixed, predictable period of time. fixed interval (FI) schedule. A schedule in which reinforcement is contingent upon the first response after a fixed, predictable period of time. fixed ratio (FR) schedule. A schedule in which reinforcement is contingent upon a fixed, predictable number of responses. fixed time (FT) schedule. A schedule in which the reinforcer is delivered following a fixed, predictable period of time, regardless of the organism’s behavior. goal gradient effect. An increase in the strength and/or efficiency of responding as one draws near to the goal. incentive motivation. Motivation derived from some property of the reinforcer, as opposed to an internal drive state. intermittent (or partial) reinforcement schedule. A schedule in which only some responses are reinforced. noncontingent schedule of reinforcement. A schedule in which the reinforcer is delivered independently of any response. Also known as a response-independent schedule. Premack principle. The notion that a high-probability (or high-frequency) behavior can be used to reinforce a low-probability (or low-frequency) behavior. ratio strain. A disruption in responding for reinforcement due to an overly demanding response requirement. response deprivation hypothesis. The notion that a behavior can serve as a reinforcer when (1) access to the behavior is restricted and (2) its frequency thereby falls below its preferred (or baseline) level of occurrence. response-rate schedule. A schedule in which reinforcement is directly contingent upon the organism’s rate of response. schedule of reinforcement. The response requirement that must be met in order to obtain reinforcement. variable duration (VD) schedule. A schedule in which reinforcement is contingent upon continuous performance of a behavior for a varying, unpredictable period of time. Chapter Test variable interval (VI) schedule. A schedule in which reinforcement is contingent upon the first response after a varying, unpredictable period of time. variable ratio (VR) schedule. A schedule in which reinforcement is contingent upon a varying, unpredictable number of responses. variable time (VT) schedule. A schedule in which the reinforcer is delivered following a varying, unpredictable period of time, regardless of the organism’s behavior. Chapter Test 21. On a(n) schedule, reinforcement is contingent upon the first response during a varying period of time. (A) fixed interval, (B) variable time, (C) fixed time, (D) variable interval, (E) none of the preceding. 6. On a(n) _________________schedule (abbreviated __________________), reinforcement is contingent upon a fixed, predictable number of responses. This produces a rate of response often accompanied by a ____________________. 17. On a (use the abbreviations) schedule, a minimum amount of time must pass between each response before the reinforcer will be delivered. On a(n) schedule, reinforcement is contingent upon emitting at least a certain number of responses in a certain period of time. On a(n) schedule, reinforcement is contingent on emitting a series of responses at a specific rate. 10. Jason regularly asks Neem out for a date, and she occasionally accepts his invitation, but only if she has nothing else to do. Of the four basic schedules, Jason’s behavior of asking Neem for a date is most likely on a schedule of reinforcement. 36. Russ is so impressed with how quickly his betta fish learned to swim in a circle that he keeps doubling the number of circles it has to perform in order to receive a reinforcer. This is an example of a(n) ____________________ schedule of reinforcement (one that is particularly likely to suffer from r____________________ s____________________). 8. On a(n) ____________________ schedule, a response must not occur until 20 seconds have elapsed since the last reinforcer. (A) VI 20-sec, (B) VT 20-sec, (C) FT 20-sec, (D) FI 20-sec, (E) none of the preceding. 28. Post-reinforcement pauses are most likely to occur on which two types of simple intermittent schedules? and ____________________. 16. On a(n) type of schedule, reinforcement is contingent upon the rate of response. 287 288 Chapter 7 Schedules and Theories of Reinforcement 31. Shawna often goes for a walk through the woods, but she rarely does yard work. According to the , walking through the woods could be used as a(n) ____________________ for yard work. 5. On a(n) schedule (abbreviated ), reinforcement is contingent upon the first response after a fixed period of time. This produces a(n) pattern of responding. 13. A(n) schedule generally produces a high rate of response with a short pause following the attainment of each reinforcer. In general, the higher the requirement, the (longer/shorter) the pause. 29. On a(n) schedule, a response cannot be reinforced until after 20 seconds have elapsed since the last reinforcer. (A) VI 20-sec, (B) VT 20-sec, (C) FT 20-sec, (D) FI 20-sec, (E) none of the preceding. 37. Ahmed’s daily routine consists of swimming without rest for 30 minutes, following which he takes a break. This most closely resembles a(n) schedule of reinforcement. 3. If a dog receives a treat each time it begs for one, its begging is being maintained on a(n) schedule of reinforcement. If it only sometimes receives a treat when it begs for one, its begging is being maintained on a(n) schedule of reinforcement. 27. Dersu often carried a lucky charm with him when he went out hunting. This may be because the appearance of game was often on a(n) (use the abbreviation) schedule of reinforcement. 32. Gina often goes for a walk through the woods, and even more often she does yard work. According to the , walking through the woods could still be used as a reinforcer for yard work given that one restricts the frequency of walking to (above/below) its level. 26. On a fixed interval schedule, reinforcement is contingent upon the first response a fixed period of time. (A) during, (B) before, (C) after, (D) none of the preceding. 9. Neem accepts Jason’s invitation for a date whenever she happens to be in the mood for a date. Of the four basic intermittent schedules, Jason’s behavior of asking Neem for a date is most likely on a(n) schedule of reinforcement. 38. When Deanna screams continuously, her parents occasionally pay attention to her. This is most likely an example of a(n) schedule of reinforcement. 30. Drinking a soda to quench your thirst is an example of ____________________ reduction; drinking a soda because you love its tangy sweetness is an example of ____________________ motivation. 4. On a(n) schedule (abbreviated ), reinforcement is contingent upon a varying, unpredictable number of responses. This generally produces a(n) rate of response (with/without) a post-reinforcement pause. Chapter Test 24. A pigeon pecks a green key on a VI 60-sec schedule, which results in the insertion of a foot-treadle into the chamber. The pigeon then presses the treadle 10 times and then receives food. To train this chain of behaviors, one should start with . 11. Neem accepts Jason’s invitation for a date only on Fridays, after she has finished work. Of the four simple schedules, the contingency governing Jason’s behavior of asking Neem for a date seems most similar to a(n) schedule of reinforcement. 35. “If I’m not a success in every aspect of my life, my family will reject me.” This is a severe example of a(n) schedule of reinforcement. 25. Dagoni works for longer and longer periods of time and takes fewer and fewer breaks as his project nears completion. This is an example of the effect. 18. On a(n) type of schedule, the reinforcer is delivered regardless of the organism’s behavior. 7. On a(n) _________________ schedule (abbreviated ), reinforcement is contingent upon the first response after a varying interval of time. This produces a ____________________ rate of response (with/without) a post-reinforcement pause. 15. Gambling is often maintained by a(n) schedule of reinforcement. 20. On a(n) schedule (abbreviated ), the reinforcer is delivered following a varying period of time. 33. Anna ideally likes to exercise for one hour each morning, sit in the sauna for 30 minutes, then spend a half hour drinking coffee and reading the newspaper. Unfortunately, due to other commitments, she actually spends 45 minutes exercising, 15 minutes in the sauna, and a half hour drinking coffee and reading the paper. According to the approach, Anna’s ideal schedule provides the amount of reinforcement that can be obtained from those activities, while her actual schedule represents her attempt to draw as near to the point as possible. 1. A(n) is the response requirement that must be met to obtain reinforcement. 22. A(n) schedule is a sequence of two or more component schedules, each of which has its own ____________________ stimulus and the last of which results in a(n) reinforcer. 34. The abbreviation DRL refers to reinforcement of rate behavior. 14. As noted in the opening scenario to this chapter, Mandy found that she had to work very hard to entice Alvin to pay attention to her. It is quite likely that her behavior was on a(n) schedule of reinforcement. As a result, she began experiencing periods of time where she simply gave up and stopped trying. Eventually, she stopped seeing 289 290 Chapter 7 Schedules and Theories of Reinforcement him altogether. When her sister asked why, Mandy, having just read this chapter, replied, “ .” 2. Different response requirements have different effects on behavior. For example, ratio schedules tend to produce (higher/lower) rates of behavior than interval schedules. Likewise, fixed schedules tend to produce , whereas variable schedules often do not. Such differences in response patterns are known as ____________________. 23. A pigeon pecks a green key on a VR 9 schedule, then a red key on an FI 20-sec, and then receives food. The reinforcer for pecking the green key is the presentation of the , which is a(n) ____________________ reinforcer. 12. Eddy finds that he has to thump his old dishwasher exactly twice before it will start rinsing. His behavior of thumping the dishwasher is on a (be specific and use the abbreviation) schedule of reinforcement. 19. On a(n) __________________ schedule (abbreviated ), the reinforcer is delivered following a fixed interval of time, regardless of the organism’s behavior. Answers to Chapter Test 1. schedule of reinforcement (reinforcement schedule) 2. higher; post-reinforcement pauses; schedule effects 3. continuous (CRF or FR1); intermittent 4. variable ratio; VR; high; without 5. fixed interval; FI; scalloped 6. fixed ratio; FR; high; postreinforcement pause 7. variable interval; VI; moderate (& steady); without 8. E (it would instead be an example of a DRP schedule) 9. variable interval (VI) 10. variable interval 11. fixed interval (or FI 1-week) 12. FR 2 13. fixed ratio; longer 14. variable ratio (VR); ratio strain 15. variable ratio 16. response rate (DRL, DRH, and!DRP) 17. DRL; DRH; DRP 18. noncontingent (or response-independent) 19. fixed time; FT 20. variable time; VT 21. E 22. chained; discriminative; terminal 23. red key; secondary 24. treadle pressing 25. goal gradient 26. C 27. VT 28. fixed interval and fixed ratio 29. D 30. drive; incentive 31. Premack principle; reinforcer 32. response deprivation hypothesis; below; baseline (or preferred) 33. behavioral bliss point; optimal (maximum); bliss 34. differential; low 35. conjunctive 36. adjusting; ratio strain 37. fixed duration (or FD 30-min) 38. VD Chapter 8 Extinction and Stimulus Control Poppea gained access to Nero, and established her ascendancy. First she used flirtatious wiles, pretending to be unable to resist her passion for Nero’s looks. Then, as the emperor fell in love with her, she became haughty, and if he kept her for more than two nights she insisted that she was married and could not give up her marriage. TACITUS, THE ANNALS OF IMPERIAL ROME Chapter Outline 8.1 Extinction Side Effects of Extinction Resistance to Extinction Spontaneous Recovery Differential Reinforcement of Other Behavior 8.2 Stimulus Control The Peak Shift Effect Multiple Schedules and Behavioral Contrast Fading and Errorless Discrimination Learning Stimulus Control: Additional Applications Stimulus Generalization and Discrimination Learning Objectives 8-1 Define operant extinction and describe how it differs from extinction in classical conditioning. 8-2 Describe the extinction burst, including each of the behavioral and emotional components. 8-3 Identify factors that affect resistance to extinction. 8-4 Explain why differential reinforcement of other behavior is often preferable to extinction alone. 8-5 Define stimulus control, including the concepts of generalization and discrimination. 291 292 Chapter 8 Extinction and Stimulus Control 8-6 Describe three types of contrast effects and how multiple schedules are used to investigate them. 8-7 Describe various applications of stimulus control for animal training and for improving studying. 8-8 Describe the basic procedure for developing a habit. 8.1 Extinction In the past few chapters, we have concentrated on strengthening operant behavior through the process of reinforcement. However, as previously noted, a behavior that has been strengthened through reinforcement can also be weakened through extinction. To reiterate, extinction is the nonreinforcement of a previously reinforced response, the result of which is a decrease in the future strength of that response. As with the classical conditioning version of extinction, the term extinction in operant conditioning refers to both a procedure and a process. The procedure of extinction is the nonreinforcement of a previously reinforced response; the process of extinction is the resultant decrease in response strength. Take, for example, a situation in which a rat has learned to press a lever for food: Lever press (R) → Food (SR) If lever pressing is no longer followed by food: Lever press (R) → No food (—) then the frequency of lever pressing will decline. The act of withholding food delivery following a lever press is the procedure of extinction, and the resultant decline in responding is the process of extinction. If lever pressing ceases entirely, the response is said to have been extinguished; if it has not yet ceased entirely, the response has been only partially extinguished. Similarly, consider a child who has learned to whine to obtain candy: Whining (R) → Candy (SR) If whining no longer produces candy: Whining (R) → No candy (—) the frequency of whining will decline. The procedure of extinction is the nondelivery of candy following the behavior, and the process of extinction is the resultant decline in the behavior. If the whining is completely eliminated, then it has been extinguished. If whining still occurs, but at a lower frequency, then it has been partially extinguished. An important, but often neglected, aspect of applying an extinction procedure is to ensure that the consequence being withheld is in fact the reinforcer that is maintaining the behavior. You might believe that the 8.1 Extinction 293 consequence of candy is reinforcing a child’s tendency to whine, when in fact it is the accompanying attention from the parent. If this is the case, and the parent continues to provide attention for whining (e.g., by arguing with the child each time they whine), then withholding the candy might have little or no effect on the behavior. Conversely, some people make the blanket assumption that children often misbehave in order to get attention, and equate the procedure of extinction with simply ignoring the occurrence of a behavior. But there are many reasons why a child might misbehave, and gaining attention is only one possibility. Hence, determining the actual reinforcer that is maintaining a behavior is a critical first step in extinguishing a behavior. An error students sometimes make is to use the term extinction when referring to any decrease in the strength of a behavior. This is incorrect. The term extinction should be used only when referring to a decrease in the future strength of a behavior that is caused by the removal of the reinforcer for that behavior. A behavior can be weakened or eliminated in several ways, extinction being only one of them. For example, we can get a rat to stop pressing a lever for food by (1) no longer giving it food when it presses the lever, (2) giving it a shock when it presses the lever, or (3) giving it lots of free food ahead of time so that it is no longer hungry. Only the first is an example of extinction; the second is an example of positive punishment; and the third is an example of an abolishing operation (specifically a satiation procedure) which reduces the value of the reinforcer. We could also stop the rat from pressing the lever by simply removing the lever from the chamber, in the same way that parents might take away a child’s iPad to stop them from playing with it. That too is not an example of extinction because it simply prevents the behavior from occurring. of a previously in the strength of response, the result of which is a(n) that response. 2. Whenever Jana’s friend Karla phoned late in the evening, she would invariably complain about her coworkers, and Jana would listen attentively. But as these phone calls became more and more frequent, Jana wondered if she was reinforcing Karla’s behavior of phoning and complaining, so she decided to screen her calls and not pick up when Karla called late in the evening. Eventually, Karla stopped phoning at that time, and they resumed a normal friendship that excluded lengthy complaints over the phone. Jana used the (procedure/process) of extinction when she stopped answering Karla’s late-evening calls, while the of extinction is the eventual cessation of such calls. 3. In carrying out an extinction procedure, an important first step is to ensure that for the the consequence being withdrawn is in fact the behavior. 4. The term extinction refers to (a decrease in the future strength of a behavior/a decrease in the future strength of a behavior because it is no longer being reinforced). Quick Quiz A 1. Extinction is the 294 Chapter 8 Extinction and Stimulus Control Side Effects of Extinction When an extinction procedure is implemented, it is often accompanied by certain side effects. It is important to be aware of these side effects because they can mislead one into believing that an extinction procedure is not having an effect when in fact it is. 1. Extinction Burst. The implementation of an extinction procedure does not always result in an immediate decrease in responding. Instead, one often finds an extinction burst, a temporary increase in the frequency and intensity of responding when extinction is first implemented. Suppose, for example, that we reinforce every fourth lever press by a rat (an FR 4 schedule of reinforcement). When extinction is implemented, the rat will initially react by pressing the lever both more rapidly and more forcefully. The rat’s behavior is analogous to our behavior when we insert money into a candy machine, press the button, and receive nothing in return. We do not just give up and walk away. Instead, we press the button several times in a row, often with increasing amounts of force. Our behavior toward the machine shows the same increase in frequency and intensity that characterizes an extinction burst. 2. Increase in Variability. An extinction procedure can also result in an increase in the variability of a behavior (Antonitis, 1951). For example, a rat whose lever pressing no longer produces food might vary the manner in which it presses the lever. If the rat typically pressed the lever with its right paw, it might now try pressing it with its left paw. If the rat usually pressed the lever in the center, it might now press it more to one side or the other. Similarly, when confronted by a candy machine that has just stolen our money, we will likely vary the manner in which we push the button, such as holding it down for a second before releasing it. And we will almost certainly try pressing other buttons on the machine to see if we can at least obtain a different selection.1 3. Emotional Behavior. Extinction is often accompanied by emotional behavior (Zeiler, 1971). The hungry pigeon that suddenly finds that key pecking no longer produces food soon becomes agitated (as evidenced, for example, by quick jerky movements and wing flapping). Likewise, people often become upset when confronted by a candy machine that does not deliver the goods. Such emotional responses are what we typically refer to as frustration. 4. Aggression. One type of emotional behavior that is particularly common during an extinction procedure is aggression. In fact, extinction procedures have been used to study aggressive behavior 1 Although we have treated them separately in this text, the increase in response variability during extinction is sometimes regarded as one aspect of an extinction burst. In other words, an extinction burst can be defined as an increase in the rate, intensity, and variability of responding following the implementation of an extinction procedure. 8.1 Extinction in animals. For example, research has shown that a pigeon whose key pecking for food is placed on extinction will reliably attack another pigeon (or model of a pigeon) that happens to be nearby (Azrin, Hutchinson, & Hake, 1966). Extinction-induced aggression (also called frustration-induced aggression) is also common in humans. People often become angry with those who block them from obtaining an important goal. For that matter, even uncooperative vending machines are sometimes attacked. 5. Resurgence. A rather unusual side effect of extinction is resurgence, the reappearance during extinction of other behaviors that had previously (sometime in the past) been effective in obtaining reinforcement (Epstein, 1985). Hull (1934), for example, trained rats to first run a 20-foot pattern through a maze to obtain food, then a 40-foot pattern. When all running was placed on extinction, the rats initially persisted with the 40-foot pattern, then returned to the 20-foot pattern before quitting. It was as though they were attempting to make the food reappear by repeating a pattern that had earlier been effective. Resurgence resembles the psychoanalytic concept of regression, which is the reappearance of immature behavior in reaction to frustration or conflict. Thus, someone whose partner largely ignores them might begin spending increasing amounts of time at their parents’ house. Faced with the lack of reinforcement in a romantic relationship, they return to a setting that once provided a rich source of reinforcement. 6. Depression. Extinction can also lead to depressive-like symptoms. For example, Klinger, Barta, and Kemble (1974) had rats run down an alleyway for food and immediately followed this with an assessment of the rats’ activity level in an open field test. Thus, each session consisted of two phases: (1) running down an alleyway for food, followed by (2)!placement in an open area that the rats could freely explore. When extinction was implemented on the alleyway task, activity in the open field test first increased to above normal (a sort of generalized extinction burst), then decreased to below normal, followed by a return to normal (refer to Figure 8.1). Klinger et al. (1974) noted that low activity is a common symptom of depression; moreover, depression is often associated with loss of reinforcement (Lewinsohn, 1974). For example, if someone dies, the people for whom that individual was a major source of reinforcement are essentially experiencing extinction, and they will likely become depressed for a period of time. One symptom of such depression is a low level of activity. The fact that a similar process occurs in rats suggests that a temporary period of depression (as evidenced by a decrease in activity) following the loss of a major reinforcer!should be regarded as a normal aspect of disengagement from that reinforcer (Klinger, 1975). These side effects of extinction can obviously be an impediment to successfully implementing an extinction procedure. Note, too, that these 295 Chapter 8 Extinction and Stimulus Control Changes in rats’ activity level in an open field test as a function of extinction on a preceding straight-alley maze task. (Source: Adapted with kind permission from Springer Science + Business Media: Animal Learning and Behavior, “Cyclic activity changes during extinction in rats: A potential model of depression,” 2, 1974, pp. 313–316, by E. Klinger, S. G. Barta, & E. D. Kemble, copyright © 1974 by the Psychonomic Society.) Figure 8.1 75 Acquisition Extinction 70 65 Mean activity level 296 60 55 50 45 Baseline 40 Low activity (“depression”) 0 12 24 36 48 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Trials side effects can be inadvertently strengthened if one suddenly gives in and provides the subject with the sought-after reinforcer. Imagine, for example, that Bobbie has learned that by begging at the supermarket, he can usually entice his mother into buying him some candy. One day, however, Bobbie’s mother decides to withhold the candy, with the result that he becomes very loud and persistent (an extinction burst) as well as emotionally upset and aggressive. If Bobbie’s mother now gives in and buys him some candy, what type of behavior has she reinforced? Obviously it is not the behavior of being polite and well mannered in the supermarket. In this way, parents sometimes inadvertently train their children to throw severe temper tantrums, a tendency that could have serious consequences if maintained later in life. After all, what the media calls “road rage”—or “air rage” when passengers become belligerent on airline flights—might, in some cases, simply be an adult version of a temper tantrum, a behavior pattern that was inadvertently established in childhood. 8.1 Extinction 2. 3. 4. 5. shopping. Unfortunately, Krissy’s father has spent all of his money on building supplies, and so he tells her that he has nothing left for a toy. Krissy might now (increase/decrease) the frequency with which she asks for a toy and ask with a (louder/softer) voice.!This process is known as an e b____________________. Krissy is also likely to ask for the toy in many different ways, because extinction of a behavior. often results in an increase in the v Krissy might also begin showing a lot of em behavior, . including ag When her father still refuses to buy her a toy, Krissy suddenly asks her dad to pick her up and carry her, something she has not asked for since she was much smaller. , or what psychoanalysts This could be an example of r . call r On the car ride home, Krissy, who never did get a toy, sits silently and sullenly stares out the window. This is not surprising, because extinction is sometimes . followed by a temporary period of d Quick Quiz B 1. Krissy asks her father to buy her a toy, as he usually does, when they are out 297 Advice for the Lovelorn Dear Dr. Dee, Why is it that I act so weird whenever I break up with a guy? One day I am intent on reestablishing the relationship, and the next day I am so angry I don’t ever want to see him again. Then I usually get all depressed and lie around in bed for days on end. What a Rollercoaster Dear What, Sounds like extinction to me. The loss of a relationship is the loss of a major reinforcer in your life. You therefore go through many of the side effects that accompany extinction. You experience an extinction burst (“intent on reestablishing the relationship”), become angry (“don’t ever want to see the guy again”), and eventually get depressed. Solution: Extinction effects are a normal part of life, so don’t expect that you shouldn’t feel something. But you might be able to moderate your feelings a bit so they are not quite so painful. In particular, stay active as much as possible and seek out alternative sources of reinforcement. And try to avoid lying in bed for days on end, as this will only further reduce the reinforcement in your life. In fact, staying in bed will make just about anyone depressed, regardless of their relationship status! Behaviorally yours, Chapter 8 Extinction and Stimulus Control Two hypothetical extinction curves. Following an initial period of reinforcement at the start of the session, the extinction procedure is implemented. This results in a brief extinction burst, followed by a decline in responding. The decline is more gradual in the top example than in the bottom example and hence illustrates greater resistance to extinction. Figure 8.2 High resistance to extinction Extinction Responses per minute SR Extinction burst Slow decline in responding Time Low resistance to extinction SR Responses per minute 298 Extinction Extinction burst Rapid decline in responding Time Resistance to Extinction Resistance to extinction is the extent to which responding persists after an extinction procedure has been implemented. A response that is very persistent is said to have high resistance to extinction, while a response that disappears quickly is said to have low resistance to extinction (refer to Figure 8.2). For example, a dog that continues to beg for food at the dinner table for 20 minutes after everyone has stopped feeding it is displaying much higher resistance to extinction than does a dog that stops begging after 5 minutes. Resistance to extinction can be affected by a number of factors, including the following: 8.1 Extinction Schedule of Reinforcement The schedule of reinforcement is the most important factor influencing resistance to extinction. According to the partial reinforcement effect, behavior that has been maintained on an intermittent (partial) schedule of reinforcement will extinguish more slowly than behavior that has been maintained on a continuous schedule. Thus, lever pressing that has been reinforced on an FR 10 schedule will take longer to extinguish than lever pressing that has been reinforced on a CRF (FR 1) schedule. Similarly, lever pressing that has been reinforced on an FR 100 schedule will take longer to extinguish than lever pressing that has been reinforced on an FR 10 schedule. Resistance to extinction is particularly strong when behavior has been maintained on a variable interval (VI) or variable ratio (VR) schedule (G.!S. Reynolds, 1975); for example, a VI 30-sec schedule will produce greater resistance to extinction than a FI 30-sec schedule, and a VR 100 schedule will produce greater resistance to extinction than an FR 100 schedule. One way of thinking about the partial reinforcement effect is that the less frequent and less predictable the reinforcer, the longer it takes the person or animal to “discover” that reinforcement is no longer available (Mowrer & Jones, 1945). For example, it obviously takes much longer for a rat to discover that reinforcement is no longer available when it has been receiving reinforcement on a VR 100 schedule than on a CRF schedule. A less mentalistic interpretation is that there is a much greater contrast between a CRF schedule and extinction than between a VR 100 schedule and extinction. On a VR 100 schedule, the rat has learned to emit many responses in the absence of reinforcement; hence, it is more persistent in its responding when an extinction procedure is implemented (E. J. Capaldi, 1966). The partial reinforcement effect helps account for certain types of annoying or maladaptive behaviors that are difficult to eliminate. Dogs that beg for food are often extremely persistent. Paradoxically, as mentioned earlier, this is sometimes the result of previously unsuccessful attempts at extinction. Imagine, for example, that all family members agree to stop feeding the dog at the dinner table. If one person nevertheless slips the dog a morsel when it is making a particularly big fuss, the begging will become both more intense and more persistent. This means that the next attempt at extinction will be even more difficult. Of course, the partial reinforcement effect also suggests a possible solution to this problem. If behavior that has been continuously reinforced is less resistant to extinction, it might help to first spend several days reinforcing each instance of begging. Then, when extinction is implemented, the dog’s tendency to beg might extinguish more rapidly (Lerman & Iwata, 1996). History of Reinforcement In general, the more reinforcers that an individual has received for a behavior, the greater the resistance to extinction. Lever pressing will extinguish more rapidly if a rat has previously earned only 10 reinforcers for lever pressing than if it has earned 100 reinforcers. Likewise, a child who has only recently picked up the habit of whining for 299 300 Chapter 8 Extinction and Stimulus Control candy should stop whining relatively quickly when the behavior is placed on extinction, as opposed to a child who has been at it for several months. From a practical perspective, this means it is much easier to extinguish an unwanted behavior, such as whining for candy, when it first becomes evident (hence the saying, “nip it in the bud”). There is, however, a limit in the extent to which further reinforcers will produce increased resistance to extinction. Furomoto (1971), for example, found that resistance to extinction for key pecking in pigeons reached its maximum after about 1,000 reinforcers. Magnitude of the Reinforcer The magnitude of the reinforcer can also affect resistance to extinction. For example, large-magnitude reinforcers sometimes result in greater resistance to extinction than small-magnitude reinforcers. Thus, lever pressing might take longer to extinguish following a training period in which each reinforcer consisted of a large pellet of food than if the reinforcer were a small pellet of food. Lever pressing might also take longer to extinguish if the reinforcer was a highly preferred food item than if it were a less-preferred food item. From a practical perspective, this means that a dog’s behavior of begging at the dinner table might extinguish more easily if you first spend several days feeding it small bites of lesspreferred morsels (Lerman & Iwata, 1996). Unfortunately, one problem with this strategy is that the effect of reinforcer magnitude on resistance to extinction is not entirely consistent. In fact, researchers sometimes find that smaller reinforcers result in greater resistance to extinction (e.g., Ellis, 1962). Degree of Deprivation Not surprisingly, the degree to which an organism is deprived of a reinforcer also affects resistance to extinction. In general, the greater the level of deprivation, the greater the resistance to extinction (Perin, 1942). A rat that is only slightly hungry will cease lever pressing more quickly than a rat that is very hungry. This suggests yet another strategy for extinguishing a dog’s tendency to beg at the table: feed the dog before the meal. Previous Experience with Extinction When sessions of extinction are alternated with sessions of reinforcement, the greater the number of exposures to extinction, the quicker the behavior will extinguish during subsequent exposures (Bullock & Smith, 1953). For example, if a rat experiences several sessions of extinction randomly interspersed with several sessions of reinforcement, it will eventually learn to stop lever pressing soon after the start of an extinction session. The rat has learned that if it has not received reinforcement soon after the start of a session, it is likely that no reinforcement will be forthcoming for the remainder of the session. Similarly, a child might learn that if they do not receive candy within the first 10 minutes of whining at the supermarket, they might as well give up for the day. This also leads to the prediction that people who have been through relationship breakups on numerous occasions will more quickly get over such breakups, when compared to people who have had fewer breakup experiences. 8.1 Extinction 301 Distinctive Signal for Extinction Extinction occurs more quickly when there is a distinctive stimulus that signals the onset of extinction. As briefly noted in Chapter 6, such a stimulus is called a discriminative stimulus for extinction and is more fully discussed later in this chapter. 2. 3. 4. 5. 6. 7. to is the extent to which responding persists after an extinction procedure is implemented. According to the p r effect, responses that have been maintained on an intermittent schedule will show (more/less) resistance to extinction than responses that have been reinforced on a continuous schedule. Among the four basic intermittent schedules, the (use the abbreviation) and schedules are particularly likely to produce strong resistance to extinction. In general, a behavior that has been reinforced many times is likely to be (much easier/more difficult) to extinguish. Resistance to extinction is generally greater when the behavior that is being extinguished has been reinforced with a (high/low)-magnitude reinforcer, though the opposite effect has also been found. In general, there is a(n) (direct/inverse) relationship between resistance to extinction and the organism’s level of deprivation for the reinforcer. Previous experience with extinction, as well as a distinctive signal for extinction, tends to produce a(n) (increase/decrease) in resistance to extinction. Spontaneous Recovery Although extinction is a reliable process for weakening a behavior, it would be a mistake to assume that once a response has been extinguished, it has been permanently eliminated. As with extinction of a classically conditioned response, extinction of an operant response is often followed by spontaneous recovery (Skinner, 1938). In this case, spontaneous recovery is the reappearance of an extinguished response, despite the continued absence of reinforcement, following a rest period after extinction. Suppose, for example, that we extinguish a rat’s behavior of lever pressing. The next day, when we place the rat back in the experimental chamber, it will probably commence lever pressing again. It is almost as though it has forgotten that lever pressing no longer produces food. Nevertheless, the behavior will likely be weaker than it was at the start of the extinction phase the day before and will extinguish more quickly given that we continue to withhold reinforcement. Similarly, on the third day, we might again find some recovery of lever pressing, but it will be even weaker than the day before and will extinguish even more quickly. This process might repeat itself several times, with each recovery usually being weaker and more readily extinguished than the previous one. Following several extinction sessions, we will eventually reach the point at which spontaneous recovery does not occur (apart from a few tentative lever presses every once in a while), and the behavior will have essentially been eliminated (refer to Figure 8.3). Likewise, a child’s tendency to throw Quick Quiz C 1. R 302 Chapter 8 Extinction and Stimulus Control Graph of hypothetical data illustrating spontaneous recovery across repeated sessions of extinction. Figure 8.3 Session 2 Session 3 Session 4 Session 5 Session 6 Responses per minute Session 1 Time Quick Quiz D tantrums in the supermarket to obtain candy might require several visits to the supermarket during which a tantrum does not produce candy before the behavior is fully eliminated. In short, when applying an extinction procedure, you have to be persistent. Skinner (1950) proposed that spontaneous recovery may be a function of discriminative stimuli (SDs) associated with the start of the session. For an experimental rat, the experience of being taken from the home cage, weighed, and placed in an operant chamber is itself a signal for the availability of food. (“Oh, goody, I’m being weighed. That means I’ll soon be able to earn some food by lever pressing.”) Only after repeated exposure to these events without receiving food does the rat at last stop emitting the learned behavior. Similarly, for the child who has learned to throw tantrums in the supermarket to receive candy, entering the supermarket is itself an SD for the availability of candy. The child will require repeated exposure to the sequence of entering the supermarket, throwing a tantrum, and not receiving candy before this cue becomes ineffective. 1. S is the reappearance of an extinguished response at a later point in time. 2. In general, each time this occurs, the behavior is (weaker/stronger) than before and extinguishes (more/less) readily. 3. Skinner believed that this phenomenon may be a function of (stimuli/responses) that are uniquely associated with the start of the session. Differential Reinforcement of Other Behavior The process of extinction can be greatly enhanced by both extinguishing the target behavior and reinforcing the occurrence of a replacement behavior. This procedure is known as differential reinforcement of other behavior (DRO), which is the reinforcement of any behavior other than the target behavior 8.1 Extinction 303 that is being extinguished. DRO procedures tend to be more effective than simple extinction procedures because the target behavior is weakened both by (1) the lack of reinforcement for that behavior and (2) the reinforcement of alternative behaviors that come to replace it. Hence, it is easier to extinguish a child’s habit of whining for candy at a supermarket if you not only withdraw the reinforcement for whining but also explicitly reinforce alternative behaviors, especially the behavior of being well-mannered.2 Moreover, in a DRO procedure, the child is not being deprived of reinforcement within that setting, which will thereby reduce or eliminate possible side effects normally resulting from extinction. Note that the reinforcement for well-mannered behavior can include the very candy for which the child has been whining. They can therefore still obtain candy, but only if they exhibit an appropriate pattern of behavior. (The candy, of course, can be gradually phased out—or replaced by a healthier treat—as the appropriate behavior becomes firmly established.) A particularly useful type of differential reinforcement procedure is called functional communication training (or differential reinforcement of functional communication). Many unwanted behaviors occur because the child is attempting to attain an important reinforcer, such as attention, but is doing so inappropriately. If the child is instead taught to communicate their need for the reinforcer in a socially appropriate manner (“Gee Dad, I’m really bored. Can you help me find something interesting to do?”), the frequency of inappropriate behaviors (such as misbehaving to get dad’s attention) is likely to decrease. Thus, in functional communication training, the behavior of clearly and appropriately communicating one’s desires is differentially reinforced (e.g., Durand, 1990). As noted, differential reinforcement procedures are not only efficient but can also reduce many of the unwanted side effects of extinction, such as frustration and aggression. As a general rule, therefore, whenever one attempts to extinguish an unwanted behavior, one should also provide plenty of positive reinforcement for more appropriate behavior (Miltenberger, 2012). r wish to extinguish is known as d behavior (abbreviated ). of o 2. Giving a dog a treat whenever it does something other than jump up on visitors as they enter the house is an example of a(n) (use the abbreviation) procedure. 3. The preceding procedure tends to (strengthen/reduce) many of the side effects of bu . extinction, such as an ex 2 A precise type of DRO procedure in which the “other behavior” is specifically incompatible with the target behavior is differential reinforcement of incompatible behavior (DRI). Thus, paying attention to a child if they are doing something other than teasing the dog is a basic DRO procedure, while paying attention to them only when they are interacting appropriately with the dog is a DRI procedure. Quick Quiz E 1. The procedure of reinforcing all behaviors except the particular target behavior that you 304 Chapter 8 Extinction and Stimulus Control And Furthermore Extinction of Bedtime Tantrums in Young Children A common difficulty faced by many parents is training children to go to bed at night without fussing or throwing a tantrum. The problem often arises because parents pay attention to a child who is throwing a tantrum and getting out of bed, thereby inadvertently reinforcing the very behavior that is annoying them. Of course, the obvious solution to this problem is for the parents to place the child’s tantrums on extinction by leaving the child alone in their room until they finally fall asleep. Research has in fact shown this to be an effective procedure. Rickert and Johnson (1988), for example, randomly assigned children to either a systematic ignoring condition (extinction), scheduled awakenings throughout the night (to comfort the child), or a control condition in which parents carried on as normal. In the systematic ignoring condition, the parents were told to initially check on their child’s safety when the child made a fuss and then ignore all further cries. Results revealed that children who underwent the extinction procedure experienced considerably greater improvement in their sleep patterns than the children in the other two conditions. Thus, extinction seems to be an effective treatment for this type of problem. Unfortunately, it suffers from a major drawback. Many parents find it impossible to totally ignore their children’s persistent heartfelt pleas during the night, especially during the initial stages of treatment when such pleas are likely to be magnified in both intensity and duration (the typical extinction burst). As a result, “graduated extinction procedures” have been devised that are more acceptable to parents and less upsetting to the child. Adams and Rickert (1989), for example, instructed parents to wait for a predetermined period of time, based on what they felt was an acceptable duration, before responding to the child’s calls. The parents were also instructed to comfort the child for only 15 seconds or less. Combined with a consistent bedtime routine, this lessstringent procedure was quite effective in helping many parents, and children, finally get a good night’s sleep (refer to Mindell, 1999, for a review). 8.2 Stimulus Control As previously noted, when a behavior has been consistently reinforced in the presence of a certain stimulus, that stimulus will begin to affect the probability of the behavior. This stimulus, known as a discriminative stimulus (SD), does not automatically elicit the behavior in the manner of a stimulus eliciting a reflex (i.e., in a cause-and-effect manner; it instead merely signals the availability of reinforcement, thereby increasing the probability that the behavior will occur. The SD therefore does not “elicit” behavior in the way that a CS elicits a CR; rather, it simply increases the likelihood of the behavior occurring—which some behaviorists indicate by saying that the SD “evokes” the behavior (e.g., Miltenberger, 2012). Thus, while a tone (CS) associated with food is said to “elicit” the response of salivation, a light (SD) 8.2 Stimulus Control that signals the availability of food for lever pressing is said to “evoke” the response of lever pressing. Such behavior is said to be under stimulus control, which means that the presence of the discriminative stimulus reliably affects the probability of the behavior (or reliably “evokes” the behavior). For example, if a 2000-Hz tone signals that lever pressing will lead to food: 2000-Hz Tone (SD): Lever press (R) → Food (SR) and the rat thus learns to press the lever only in the presence of the tone, the behavior of lever pressing is said to be under stimulus control. Similarly, the sound of a ringing phone has strong stimulus control over whether people will answer it and say hello. People never answer phones that are not ringing, but often answer phones that are ringing. Here are some other examples of stimulus control (with the SD italicized): • At red traffic lights, we stop; at green traffic lights, we proceed. • hen someone says “Thank you,” we respond with “You’re welcome.” • In an elevator, we stand facing the front rather than the back. • hen we hear an ambulance siren behind us, we pull our car over to the side of the road and stop or slow down. • hen you receive a notification on your cell phone, you immediately read it (despite promising yourself that you wouldn’t look at your phone while studying). • hen the professor begins lecturing, students cease talking among themselves.3 In this section, we will look more closely at discriminative stimuli and their effects on behavior. Note that some of the principles discussed, such as stimulus generalization, represent operant versions of principles discussed in earlier chapters on classical conditioning. Stimulus Generalization and Discrimination In our discussion of classical conditioning, we noted that stimuli that are similar to a CS can also elicit a CR, by a process known as stimulus generalization. A similar process occurs in operant conditioning. In operant conditioning, stimulus generalization is the tendency for an operant response to be emitted in the presence of a stimulus that is similar to an SD. In general, the more similar the stimulus, the stronger the response. Take, for example, a rat that has learned to press a lever for food whenever it hears a 2000-Hz tone. If we present the rat with a series of tones that vary in pitch, we will find that it also presses the lever in the presence of these other tones, particularly in the presence of a tone that is similar to the original SD. Thus, the rat will display a higher rate of lever pressing in the presence of an 1800- or 2200-Hz 3 Some of these examples also represent a special type of stimulus control known as an instructional control (“Do not drive through red lights, or you will get a ticket!”). The concept of instructional control is discussed in the section on rule-governed behavior in Chapter 11. 305 Chapter 8 Extinction and Stimulus Control tone, both of which are more similar to the original SD, than in the presence of a 1200- or 2800-Hz tone, which are less similar. This tendency to generalize across different stimuli is called a generalization gradient, which represents the strength of responding in the presence of stimuli that are similar to the SD and that vary along a continuum. As shown in Figure 8.4, gradients can vary in their degree of steepness. A relatively steep gradient indicates that rate of responding drops sharply as the stimuli become increasingly different from the SD, while a relatively flat Figure 8.4 Two hypothetical generalization gradients depicting rate of lever pressing in the presence of tones that vary in pitch between 1200 and 2800 Hz (“Hertz” is the number of sound waves per second generated by a sound source). In both examples, tones that are more similar to the original SD (a 2000-Hz tone) are associated with stronger responding. However, generalization is much greater in the bottom gradient, which is relatively flat, than in the top gradient, which is relatively steep. Relatively steep generalization gradient Lever presses: responses per minute 306 1200 1400 1600 Strength of responding D to the original S 1800 2000 2200 2400 2600 2800 Relatively flat generalization gradient Strength of responding D to the original S 1200 1400 1600 1800 2000 2200 Tonal pitch (in Hertz) 2400 2600 2800 8.2 Stimulus Control 307 gradient indicates that responding drops gradually as the stimuli become increasingly different from the SD. In other words, a flat gradient indicates more generalization, while a steep gradient indicates less generalization.4 As in classical conditioning, the opposite of stimulus generalization in operant conditioning is stimulus discrimination, the tendency for an operant response to be emitted more in the presence of one stimulus than another. More generalization means less discrimination, and less generalization means more discrimination. Thus, a steep gradient indicates weak generalization and strong discrimination across stimuli, whereas a flat gradient indicates strong generalization and weak discrimination across stimuli. when it is highly likely to occur in the presence of a certain stimulus. 2. In operant conditioning, the term s___________________ g____________________ 3. 4. 5. 6. 7. refers to the tendency for a response to be emitted in the presence of stimuli that . The opposite process, called are similar to the original s____________________ d____________________, refers to the tendency for the response to be emitted more in the presence of one stimulus than another. In general, stimuli that are (more/less) similar produce stronger generalization. Ag gr indicates the strength of responding to stimuli that vary along a continuum. In a graph that depicts a g__________________ g__________________, a relatively flat line indicates more __________________ and less __________________. A relatively steep line indicates more __________________ and less ___________________. When Jonathan looked at his watch and saw that it was 12:30 P.M., he decided that it was time for lunch. Jonathan’s eating behavior appears to be under strong s____________________ c____________________. Jonathan always goes for lunch at about 12:30, with a range somewhere between 12:25 and 12:35 P.M. The generalization gradient for this behavior across various points in time would therefore be much (steeper/flatter) than if the range were between 12:00 and 1:00. This indicates a pattern of strong (discrimination/ for Jonathan’s lunch-going generalization) and weak behavior across different points in time. Discrimination training, as applied to operant conditioning, involves the reinforcement of responding in the presence of one stimulus (the SD) and not another stimulus. As noted in Chapter 6, the latter is called a discriminative stimulus for extinction (S"), which is a stimulus that signals the absence of reinforcement. For example, if we wish to train a rat to discriminate between a 2000-Hz tone and a 1200-Hz tone, we would present the two tones in random order. Whenever the 2000-Hz tone sounds, a lever press 4 Generalization gradients are also used to indicate the extent of stimulus generalization in classical conditioning. Imagine, for example, that the 2000-Hz tone in Figure 8.4 is a CS that has been associated with food and now elicits a conditioned salivary response. A steep generalization gradient would indicate weak generalization of the CR across tones, while a flat gradient would indicate strong generalization of the CR across tones. Quick Quiz F 1. A behavior is said to be under strong s_________________ c________________ 308 Chapter 8 Extinction and Stimulus Control produces food; whenever the 1200-Hz tone sounds, a lever press does not produce food. 2000-Hz Tone (SD): Lever press (R) → Food (SR) 1200-Hz Tone (S!): Lever press (R) → No food (—) Quick Quiz G After repeated exposure to these contingencies, the rat will soon learn to press the lever in the presence of the 2000-Hz tone and not in the presence of the 1200-Hz tone. We can then say that the rat’s behavior of lever pressing is under strong stimulus control. In similar fashion, if the manager where you work complies with your request for a day off only when they appear to be in a good mood and does not comply when they appear to be in a bad mood, you learn to make requests only when they are in a good mood. The manager’s appearance exerts strong stimulus control over the probability of your making a request. In this sense, one characteristic of people who have good social skills is that they can make fine discriminations between social cues—such as facial expression and body posture—which enables them to maximize the amount of social reinforcement (and minimize the amount of social punishment) obtained during their exchanges with others. Likewise, college roommates are more likely to live in harmony to the extent that they learn to discriminate between each other’s social cues and modify their actions appropriately. The ability to discriminate can also be tested using a matching-tosample procedure in which an animal is first shown a sample stimulus and then is required to select that stimulus out of a group of alternative stimuli. The extent to which the animal is able to select the correct stimulus is regarded as an indicator of its ability to discriminate between the two stimuli. 1. In a discrimination training procedure, responses that occur in the presence of are reinforced, while those that the (use the symbol) are not reinforced. This occur in the presence of the s latter stimulus is called a d . for e 2. An “Open for Business” sign is a(n) (use the abbreviation) for entering the store and making a purchase, while a “Closed for Business” sign is for attempting such behavior. a(n) The Peak Shift Effect An unusual effect often produced by discrimination training is the peak shift effect. According to the peak shift effect, the peak of a generalization gradient following discrimination training will shift from the SD to a stimulus that is further removed from the S" (Hanson, 1959). This constitutes an exception to the general principle that the strongest response in a generalization gradient occurs in the presence of the original SD. 8.2 Stimulus Control Suppose, for example, that we first train a rat to press a lever in the presence of a 2000-Hz tone. We then conduct a test for generalization across a range of tones varying in pitch between 1200 and 2800 Hz, and we find a generalization gradient like that shown in the top panel of Figure 8.5. We then submit the rat to a discrimination training procedure in which we reinforce lever pressing in the presence of a 2000-Hz tone (SD) and not in the presence of a 1200-Hz tone (S"). When this has been successfully accomplished (i.e., the rat responds only in the presence of the 2000-Hz tone and not in the presence of the 1200-Hz tone), we again test for generalization across a Figure 8.5 Illustration of a peak shift effect following discrimination training. Prior to discrimination training (top panel), the gradient is relatively flat. Following discrimination training (bottom panel), in which a 1200-Hz tone has been established as an S", the strongest response occurs not in the presence of the SD (the 2000-Hz tone), but in the presence of a stimulus further removed from the S". The gradient in the bottom panel therefore illustrates the peak shift effect. Lever presses: responses per minute Generalization gradient prior to discrimination training 1200 1400 1600 Strongest response D occurs to S 1800 2000 2200 Generalization gradient following discrimination training 2400 2600 2800 Strongest response Strength of response to SD 1200 1400 1600 1800 2000 2200 Tonal pitch (in Hertz) 2400 2600 2800 309 310 Chapter 8 Extinction and Stimulus Control range of tones. What we are likely to find with this rat is a generalization gradient something like that depicted in the bottom panel of Figure 8.5. Look carefully at this gradient. How does it differ from the gradient in the top portion of the figure, which represents generalization in the absence of discrimination training? One obvious difference is that, with discrimination training, the gradient drops off more sharply on the side toward the S", which simply means that this rat strongly discriminates between the S" and the SD. But what is the other difference between the two graphs? Before discrimination training (the top panel), the strongest response occurs to the SD (the 2000-Hz tone). Following discrimination training (the bottom panel), the strongest response shifts away from the SD to a stimulus that lies in a direction opposite to the!S" (in this case, it shifts to a 2200-Hz tone). This shift in the peak of the generalization gradient is what is called the peak shift effect. Perhaps a fanciful example will help clarify the peak shift effect. Suppose that “Mr. Shallow” categorizes potential dates entirely on the basis of how extraverted versus introverted they are. Jackie, with whom he had a very boring relationship, was an introvert (an S"), while Dana, with whom he had a wonderfully exciting relationship, was an extravert (an SD). He then moves to a new city and begins scrolling through dating websites to find someone new. According to the peak shift effect, he will likely seek out a woman who is even more extraverted than Dana. One explanation for the peak shift effect is that during discrimination training, subjects respond in terms of the relative, rather than the absolute values, of stimuli (Kohler, 1918/1939). Thus, according to this interpretation, the rat does not learn merely that a 2000-Hz tone indicates food and a 1200-Hz tone indicates no food; rather, it learns that a higher-pitched tone indicates food and a lower-pitched tone indicates no food. Given a choice, the rat therefore emits the strongest response in the presence of a tone that has an even higher pitch than the original SD. Likewise, Mr. Shallow chooses a woman who is even more extraverted than Dana because greater extraversion has been associated with a better relationship. Another explanation for the peak shift effect is that, despite discrimination training, the SD is still somewhat similar to the S" and has acquired some of its negative properties (Spence, 1937). From this perspective, the 2000-Hz tone (the SD) is somewhat similar to the 1200-Hz tone (the S"), making the 2000-Hz tone slightly less attractive than it would have been if the S " had never been trained. Thus, a tone that has a slightly higher pitch than 2000 Hz, and is thereby less similar to the 1200-Hz tone, will result in the highest rate of responding. Likewise, Mr. Shallow seeks a woman who is very extraverted because he is attempting to find a woman who is even more dissimilar from Jackie, with whom he had such a poor relationship.5 5 The peak shift effect is also found in classical conditioning following discrimination training between a CS+ and a CS#. For example, if the CS+ for a CR of salivation was a 2000-Hz tone and the CS# was a 1200-Hz tone, what would the peak shift effect consist of? 8.2 Stimulus Control d_______________ t__________________, shifts away from the _________________ . to a stimulus that is further removed from the 2. If an orange key light is trained as an SD in a key pecking task with pigeons, and the pigeons are then exposed to other key colors ranging from yellow on one end of the continuum to red on the other (with orange in the middle), the peak of the generalization gradient will likely be to a (yellowish-orange/orange/orangereddish) key light. 3. If a pigeon undergoes discrimination training in which a yellow key light is explicitly established as an S! and an orange key light is explicitly established as the SD, the strongest response in the generalization gradient will likely be to a (yellowish-orange/orange/orange-reddish) key light. This effect is known as the effect. Multiple Schedules and Behavioral Contrast Stimulus control also plays a role in a type of complex schedule known as a multiple schedule. A multiple schedule consists of two or more independent schedules presented in sequence, each resulting in reinforcement and each having a distinctive S D. For example, a pigeon might first be presented with a red key that signals an FI 30-sec schedule, completion of which results in food. The key light then changes to green, which signals a VI 60-sec schedule, completion of which also results in food. These two schedules can be presented in either random or alternating order, or for set periods of time (such as two minutes on the red FI 30-sec schedule followed by two minutes on the green VI 60-sec schedule followed by another two minutes on the red FI 30-sec schedule, etc.). The following schematic shows the two schedules presented in alternating order, and continuing to repeat, one after the other (leaving out the abbreviations for each component): Red: PeckFI 30” → Food/Green: PeckVI 60” → Food/Red: PeckFI 30” → . . . Note that a multiple schedule differs from a chained schedule in that a chained schedule requires that all of the component schedules be completed before the sought-after reinforcer is delivered. For example, on a chain FI 30-sec VI 60-sec schedule, both the FI and VI components must be completed to obtain food. On a multiple FI 30-sec VI 60-sec schedule, however, completion of each component schedule results in food. On a multiple schedule (as well as a chained schedule), stimulus control is demonstrated when the subject responds differently in the presence of the SDs associated with the different schedules. For example, with sufficient experience on a multiple FI 30-sec VI 30-sec schedule, a pigeon will likely show a scalloped pattern of responding on the red key signaling the FI component and a moderate, steady pattern of responding on the green key signaling the VI component. The pigeon’s response pattern on each key color will be the appropriate pattern for the schedule of reinforcement Quick Quiz H 1. In the peak shift effect, the peak of a generalization gradient, following 311 312 Chapter 8 Extinction and Stimulus Control Quick Quiz I that is in effect on that key. Similarly, if you are taking more than one class in a semester, you may have noticed that different professors have different ways of responding to questions in class. Perhaps in your morning class the professor responds to each raised hand, regardless of when it is raised (which can be considered a CRF, or FR 1, schedule of reinforcement for hand-raising). In your afternoon class, however, that professor answers questions only after about 10 minutes have passed since answering the last question (a roughly FI 10-min schedule) and ignores any raised hands until then. As a result, the likelihood of you raising your hand to ask a question will be affected by the different schedules of reinforcement in each class—that is, in the first class, you will raise your hand whenever you happen to have a question, and in the second class, you will be most likely to raise your hand after around 10 minutes has elapsed since the last question was answered. 1. On a m schedule, two or more schedules are presented (sequentially/simultaneously), with each resulting in a(n) . and having its own distinctive 2. This type of schedule differs from a chained schedule in that a(n) is provided after each component schedule is completed. 3. On a multiple VI 50-sec VR 50 schedule, we are likely to find a high rate of response on the (VI/VR/both) schedule. An interesting phenomenon that can be investigated using multiple schedules is behavioral contrast. Behavioral contrast occurs when a change in the rate of reinforcement on one component of a multiple schedule produces an opposite change in the rate of response on another component (G. S. Reynolds, 1961). In other words, as the rate of reinforcement on one component changes in one direction, the rate of response on the other component changes in the opposite direction. There are two basic contrast effects: positive and negative. In a negative contrast effect, an increase in the rate of reinforcement on one component produces a decrease in the rate of response on the other component. Suppose, for example, that a pigeon first receives several sessions of exposure to a multiple VI 60-sec VI 60-sec schedule (leaving out the abbreviations): Red: PeckVI 60” → Food/Green: PeckVI 60” → Food/ Red: PeckVI 60” → . . . Because both schedules are the same, the pigeon responds equally on both the red key and the green key. Following this, the VI 60-sec component on the red key is changed to VI 30-sec, which provides a higher rate of reinforcement (on average, two reinforcers per minute as opposed to one reinforcer per minute): Red: Peck*VI 30”* → Food/Green: PeckVI 60” → Food/ Red: Peck*VI 30”* → . . . 8.2 Stimulus Control With more reinforcement now available on the red key, the pigeon will decrease its rate of response on the green key, which is associated with the unchanged VI 60-sec component. Simply put, because the first component in the sequence is now more attractive, the second component seems relatively less attractive. The situation is analogous to someone whose partner has suddenly become much more affectionate and caring at home; as a result, they spend less time flirting with people at work. The people at work seem relatively less attractive compared to their sweetheart at home. In a positive contrast effect, a decrease in rate of reinforcement on one component results in an increase in rate of response on the other component. If, for example, on a multiple VI 60-sec VI 60-sec schedule: Red: PeckVI 60” → Food/Green: PeckVI 60” → Food/ Red: PeckVI 60” → . . . the first VI 60-sec component is suddenly changed to VI 120-sec: Red: Peck*VI 120”* → Food/Green: PeckVI 60” → Food/ Red: Peck*VI 120”* → . . . the pigeon will increase its rate of response on the unchanged VI 60-sec component. As one component becomes less attractive (changing from VI 60-sec to VI 120-sec), the unchanged component becomes relatively more attractive. The situation is analogous to the person whose partner has become less caring and affectionate at home; as a result, they spend more time flirting with people at work. The people at work seem relatively more attractive compared to the dud they have at home. Positive contrast effects are also evident when the change in one component of the multiple schedule involves not a decrease in the rate of reinforcement but instead the implementation of a punisher, such as a mild electric shock. As the one alternative suddenly becomes punishing, the remaining alternative, which is still reinforcing, is viewed as even more attractive (Brethower & Reynolds, 1962). This might explain what happens in some volatile relationships in which couples report strong overall feelings of affection for each other (Gottman, 1994). The intermittent periods of aversiveness seem to heighten the couple’s appreciation of each other during periods of affection. Such relationships can therefore thrive, given that the positive aspects of the relationship significantly outweigh the negative aspects.6 6 Contrast effects can also occur when there is a shift in the magnitude of a reinforcer (Crespi, 1942). For example, rats that experience a sudden switch from receiving a small amount of food for running down an alleyway to receiving a large amount of food for running down the alleyway will run faster for the large amount (a positive contrast effect) than if they had always received the large amount. And those that are shifted from a large amount of food to a small amount will run slower (a negative contrast effect). 313 314 Chapter 8 Extinction and Stimulus Control Warning Quick Quiz J Remember that with positive and negative contrast, we are concerned with how changing the rate of reinforcement on the first component of a multiple schedule affects the rate of responding on the second component. The rate of responding will, of course, also change on the first component because the schedule of reinforcement on that component has changed, but that is not surprising. What is surprising is the change in response rate on the second component, even though the schedule of reinforcement in that component has remained the same. Thus, it is the change in response rate on the second component that is the focus of concern in behavioral contrast. 1. In (positive/negative) behavioral contrast, an increase in (responding/ reinforcement) on one alternative results in a(n) (increase/decrease) in (responding/reinforcement) on the other alternative. 2. In ____________________ behavioral contrast, a decrease in reinforcement on one alternative results in a(n) ____________________ in ____________________ on the other alternative. 3. A pigeon that experiences a shift from a multiple FR 10 VI 60-sec schedule to a multiple FR 100 VI 60-sec schedule will likely (increase/decrease) its responding on the VI 60-sec component. This is an example of a contrast effect. 4. When Levin (a lonely bachelor in Tolstoy’s novel Anna Karenina) proposed to the beautiful young Kitty, she rejected him. Levin was devastated and decided to devote the rest of his life to his work. Kitty, in turn, was subsequently rejected by the handsome young military officer, Vronsky, whom she had mistakenly assumed was intent on marrying her. Kitty was devastated and deeply regretted having turned down Levin, whom she now perceived to be a fine man. A year later, they encountered each other at a social gathering. Relative to individuals who have not experienced such hardships in establishing a relationship, we would expect their affection for each other to be much (deeper/shallower) than normal. Of the two types of contrast effects you have just learned about this, this example is most similar to a ____________ contrast effect. An additional type of contrast effect is anticipatory contrast, in which the rate of response varies inversely with an upcoming (“anticipated”) change in the rate of reinforcement (B.!A. Williams, 1981). For example, Pliskoff (1963) found that pigeons increased their rate of responding for reinforcement when they were presented with a stimulus signaling that extinction was imminent. In other words, faced with the impending loss of reinforcement, the pigeons responded all the more vigorously for reinforcement while it was still available. Anticipatory contrast seems analogous to what many of us have experienced—that things we are about to lose often seem to increase in value. For example, Lindsay views her relationship with Bryce as rather dull and uninteresting until she learns that Bryce might be romantically interested in 8.2 Stimulus Control 315 another woman. Faced with the possibility that she might lose Bryce, she now becomes intensely interested in her. Unfortunately, some people may use anticipatory contrast as a deliberate tactic to strengthen a partner’s feelings of attachment without considering any long-term effects on the quality of the relationship. Read again the anecdote at the beginning of this chapter about Poppea’s relationship with the Roman emperor Nero. In behavioral terms, Poppea first established herself as an effective reinforcer for Nero; then, to further increase her value, she intermittently threatened to withdraw herself from his company. In anticipation of possibly losing her, Nero became even more attached. The occurrence of these contrast effects indicates that behaviors should not be viewed in isolation. Consequences for behavior in one setting can greatly affect the strength of behavior in another setting. Consider, for example, a young girl who is increasingly neglected at home, perhaps because her parents are going through a divorce. She might try to compensate for this circumstance by seeking more attention at school (a positive contrast effect), perhaps to the point of misbehaving. Although her parents might blame the school for her misbehavior, she is in fact reacting to the lack of reinforcement at home. Thus, to borrow a concept from humanistic psychology, behavior needs to be viewed in a holistic manner, with the recognition that behavior in one setting can be influenced by contingencies operating in other settings. the possibility of losing it in the near future is known as contrast. 2. If Jackie hears her mother say that it is getting close to her bedtime, she is likely involved in the computer game to become (more/less) she is playing. 3. Vronsky (a character in Tolstoy’s Anna Karenina) falls deeply in love with Anna, who is the wife of another man. For several months, they carry on a passionate affair. When Anna, however, finally leaves her husband to be with him, Vronsky finds that he soon becomes bored with their relationship. The fact that his feelings for Anna were much stronger when their relationship was more precarious is in contrast effect. keeping with the Fading and Errorless Discrimination Learning While discrimination training is an effective way for establishing stimulus control, it has its limitations. For example, during the process of learning to discriminate an SD from an S!, the subject will initially make several “mistakes” by responding in the presence of the S!. Because such responses do not result in reinforcement, the subject may become frustrated and display a great deal of emotional behavior. It would be helpful, therefore, if there were a method of discrimination training that minimized these effects. Quick Quiz K 1. An increase in the rate of responding for an available reinforcer when faced with 316 Chapter 8 Extinction and Stimulus Control And Furthermore St. Neots Margin The anticipatory contrast effect described by Pliskoff (1963) reflects the pigeon’s reaction to a potential difficulty—namely, the impending loss of a reinforcer. According to British writer Colin Wilson (1972), such difficulties can provide our lives with a sense of meaning when more pleasant stimuli have failed. Wilson’s description of how he discovered this concept provides an interesting illustration. In 1954, I was hitch-hiking to Peterborough on a hot Saturday afternoon. I felt listless, bored and resentful: I didn’t want to go to Peterborough—it was a kind of business trip—and I didn’t particularly long to be back in London either. There was hardly any traffic on the road, but eventually I got a lift. Within ten minutes, there was an odd noise in the engine of the lorry. The driver said: ‘I’m afraid something’s gone wrong—I’ll have to drop you off at the next garage.’ I was too listless to care. I walked on, and eventually a second lorry stopped for me. Then occurred the absurd coincidence. After ten minutes or so, there was a knocking noise from his gearbox. When he said: ‘It sounds as if something’s gone wrong,’ I thought: ‘Oh no!’ and then caught myself thinking it, and thought: ‘That’s the first definite reaction I’ve experienced today.’ We drove on slowly—he was anxious to get to Peterborough, and by this time, so was I. He found that if he dropped speed to just under twenty miles an hour, the knocking noise stopped; as soon as he exceeded it, it started again. We both listened intently for any resumption of the trouble. Finally, as we were passing through a town called St. Neots, he said: ‘Well, I think if we stay at this speed, we should make it.’ And I felt a surge of delight. Then I thought: ‘This is absurd. My situation hasn’t improved since I got into the lorry—in fact, it has got worse, since he is now crawling along. All that has happened is that an inconvenience has been threatened and then the threat withdrawn. And suddenly, my boredom and indifference have vanished.’ I formulated then the notion that there is a borderland or threshold of the mind that can be stimulated by pain or inconvenience, but not pleasure. (p. 27) Wilson labeled the concept St. Neots margin after the town they were driving through at the time. He proposes that such difficulties create “meaning” by forcing us to concentrate, and that the absence of such concentration makes life dull and uninteresting. But we can also view these difficulties as a type of contrast effect in which we are in danger of losing a reinforcer. As a result, we respond more vigorously for the reinforcer and value it more highly. Contrast effects may therefore provide our lives with a sense of meaning that might otherwise be missing. Wilson describes, for example, how the writer Sartre claimed that he never felt so free as during the war when, as a member of the French Resistance, he was in constant danger of being arrested. In danger of losing his freedom, he truly appreciated his freedom. Consider too Balderston’s (1924) play A Morality Play for the Leisured Class, which recounts the story of a man who dies and finds himself in the afterlife. When a shining presence tells him that he can have any pleasure he desires by merely wishing it, he is overjoyed and fully indulges himself. He soon discovers, however, that things quickly lose their value when they are so easily attained. Facing an eternity of profound boredom (in which contrast effects are completely absent), he finally exclaims that he would rather be in hell—at which point the presence asks: “And wherever do you think you are, sir?” 8.2 Stimulus Control Errorless discrimination training is a gradual training procedure that minimizes the number of errors (i.e., nonreinforced responses to the S") and reduces many of the adverse effects associated with discrimination training. It involves two aspects: (1) The S" is introduced early in training, soon after the animal has learned to respond appropriately to the SD, and (2) the S" is presented in weak form to begin with and then gradually strengthened. This process of gradually altering the intensity of a stimulus is known as fading. (For example, one can fade in music by presenting it faintly to begin with and gradually turning up the volume, or fade out music by presenting it loudly to begin with and gradually turning down the volume.) Terrace (1963a) used errorless discrimination training to establish a red-green discrimination in pigeons. The pigeons were first trained to peck a red key on a VI 60-sec schedule of reinforcement. As soon as this behavior was established, occasional 5-second periods of extinction were presented in which the key light was switched off. Since pigeons tend not to peck a dark key, the dark key was easily established as an effective S" for not responding. The VI period and the extinction period were gradually lengthened until they each lasted three minutes. Following this, the dark key was illuminated with a faint greenish hue that was slowly intensified. As the green key color was faded in (as an S " ) and gradually replaced the dark key, the pigeons emitted almost no responses toward it; that is, they made almost no errors. By comparison, pigeons that were exposed to a dark key that was suddenly replaced by a brightly lit green key made numerous responses on it before finally discriminating it from the red SD key. The pigeons exposed to the errorless procedure also showed few of the adverse side effects of discrimination training, such as emotional behavior. Errorless procedures can also be used to transfer control from one type of stimulus to another. For example, Terrace (1963b) first trained pigeons to discriminate between a red key as the SD and a green key as the S ". He gradually faded in a vertical line (the new SD) on the red key and a horizontal line (the new S") on the green key, while at the same time fading out the colors. Eventually, the pigeons were pecking a colorless key that had a vertical line and not pecking a colorless key that had a horizontal line. With virtually no errors, stimulus control for pecking had been transferred from key color (red versus green) to line orientation (vertical versus horizontal). Errorless discrimination training may have practical applications. For example, Haupt, Van Kirk, and Terraciano (1975) used an errorless procedure to enhance the learning of basic arithmetic skills. In their study, a nine-year-old girl who had a history of difficulties in basic arithmetic was given a series of addition problems using a standard drill procedure and a series of subtraction problems using an errorless procedure. The standard drill procedure for the addition problems consisted of 317 318 Chapter 8 Extinction and Stimulus Control Quick Quiz L presenting the problems on flash cards in which the answers were initially covered. If the child did not know the answer, the answer was uncovered and shown to her. The errorless procedure for the subtraction problems was similar except that the answer on each flash card was initially left exposed to view and, over successive presentations, gradually blocked out by adding successive sheets of cellophane. The correct answer was thus initially available as a prompt for the correct answer and then gradually faded out. During a subsequent test, the girl made significantly fewer errors on the subtraction problems, for which the errorless procedure had been used, than on the addition problems, for which the standard drill procedure had been used. Although errorless discrimination training might seem like the perfect answer to many unresolved problems in education, it has some serious drawbacks. Discriminations that have been established through errorless training are more difficult to modify at a later time. For example, Marsh and Johnson (1968) taught pigeons to discriminate between two key colors: one was the S D and the other the S ". Pigeons that had been taught to discriminate using an errorless procedure experienced extreme difficulty learning a new discrimination when the meaning of the key colors was reversed (i.e., the color that had previously been the S " now became the SD, and vice versa). In contrast, pigeons that had learned the original discrimination in the normal error-filled way handled the reversal quite handily. Thus, although normal discrimination training has more adverse side effects compared to errorless discrimination training, it also results in greater flexibility when what is learned has to be modified later. For this reason, errorless procedures may be most useful in rote learning of basic facts, such as arithmetic and spelling, in which the substance of what is learned is unlikely to change. With material that requires greater flexibility, however, such as that typically found in most college-level courses, errorless learning might be a significant impediment (Pierce & Epling, 1999).7 discrimination training, the S" is presented (early/ later) in the training procedure, and at very (weak/strong) intensity to begin with. 2. This type of discrimination training is likely to produce (more/less) emotional behavior compared to the standard form of discrimination training. 3. This type of discrimination training is also likely to produce behavior patterns that are (easy/difficult) to modify at a later point in time. 4. Gradually altering the intensity of a stimulus is called fa . 1. In e 7 This accords with the more general finding, briefly mentioned in Chapter 1, that experiencing a certain amount of difficulty during the learning process can enhance long-term retention and understanding (Schmidt & Bjork, 1992). 8.2 Stimulus Control Stimulus Control: Additional Applications There are many ways in which stimulus control can be used to manage behavior. Perhaps the most impressive use of stimulus control is by animal trainers, especially those who train animals for public performance. Dolphin trainers, for example, use a mere whistle or gesture to set off a dazzling display of leaps and twirls. Indeed, the control is so precise that the dolphins often seem like robots, an impression that probably contributes to the growing opposition to such shows. Not only has the animal been removed from its natural environment, it now appears to be a slave to the trainer’s every whim. (Karen Pryor, 1999, however, contends that the reality is quite different, with such training—especially to the extent that it involves positive reinforcement—being much more a two-way process of communication than brute force control.) A particularly useful form of stimulus control for animal management is targeting. In targeting, one trains an animal to approach and touch a particular object, as in training a dog to touch the end of a stick with its nose. Targeting is a key aspect of teaching dolphins to make their impressive leaps. The dolphin first receives reinforcement for touching a target stick with its snout, then the stick is raised higher and higher, enticing the dolphin to leap higher and higher to touch it. Targeting is commonly used to manage animals in zoos. By simply moving the target stick, zookeepers can lead the animals from one cage to another or position them precisely for medical examinations. Animals can also be taught to target a point of light from a laser beam, which allows the handler to send the animal to a spot some distance away. This can be a useful procedure for directing search-and-rescue dogs in disaster areas that are difficult for the handler to traverse (Pryor, 1999). Search-and-rescue dogs are, of course, an excellent example of stimulus control in animals that can be of considerable benefit for humans. Dogs’ keen sense of hearing and smell has proven useful for a variety of tasks, which also includes drug detection and even landmine detection. In an interesting twist on the latter, giant African pouched rats have also been trained for landmine detection (Poling, Weetjens, Cox, Beyene, & Sully, 2010; Jones, 2011). One of the several advantages these rats have over dogs is that they are so light that they can step on a landmine without setting it off! Another area in which discrimination training with animals may prove useful is the cancer clinic. Interest began in 1989 with a letter to a medical journal that described a woman whose dog persistently sniffed at a certain mole (Williams & Pembroke, 1989). When she had it checked out by a physician, it was found to be malignant. Several studies have now been conducted on this topic, and results have so far been encouraging (Pirrone & Albertini, 2017). Research is also underway to determine if dogs are capable of detecting viral infections, including COVID-19 (Jendrny et al., 2020). 319 320 Chapter 8 Extinction and Stimulus Control Giant African pouched rats are likewise getting in on the action, only in this case with detection of tuberculosis by sniffing samples of patients’ sputum (Poling et al., 2015). But what about the behavior analyst’s favorite animal, the pigeon? Well, pigeons do have excellent vision, and behavior analysts have tried to make use of this ability. In WWII, Skinner proposed using pigeons housed in the nose cone of a missile to visually guide the missile to a military target. He was in fact able to demonstrate that pigeons were capable of performing this task, but the military was unenthusiastic about it and the project was dropped. Similarly, in the 1970s, an attempt was made to train pigeons for search-andrescue missions at sea (Azar, 2002). Housed in a transparent bubble below a helicopter, the pigeons responded to the sight of tiny objects in the seascape by pecking at response keys that guided the pilot to the target. The pigeons were reportedly 93% accurate—the flight crews were only 38% accurate— but the program was nevertheless cancelled when the need for budget cuts arose. As with Skinner’s missile project, it seems that the “laugh factor” may have also played a role; it was difficult for people to take these pigeon projects seriously. Stimulus control can also be used to eliminate certain types of problem behaviors in animals. Pryor (1999), for example, describes how she once experienced considerable difficulty in training a dolphin to wear suction cups over its eyes (as part of an intended demonstration of the dolphin’s ability to swim solely by sonar). Although the cups did not hurt, the dolphin refused to wear them and would cleverly sink to the bottom of the pool for several minutes whenever it saw Pryor approaching with the cups. Initially stumped, Pryor finally hit on the idea of reinforcing the behavior of sinking by giving the dolphin a fish whenever it did so (which, she reports, seemed to greatly surprise the dolphin). Soon, the dolphin was sinking at high frequency to earn fish, at which point Pryor began to reinforce the behavior only after a cue had been presented. In short order, the dolphin was sinking only on cue, meaning that the behavior was now under strong stimulus control. Pryor found that she was able to reintroduce the suction cups and place them on the dolphin without difficulty. In the absence of the cue for sinking, the dolphin no longer had a tendency to sink to avoid the cups. In similar fashion, a dog that has been trained to bark on cue may be less likely to bark at other times. In short, by putting a behavior “on cue,” the behavior is less likely to occur in the absence of the cue. Stimulus control is also an important aspect of human behavior, one that is often overlooked as a means of improving behavior. Consider Stephanie, who promises herself that she will take vitamins each evening but forgets so often that she eventually gives up. All she might need to do is create a salient cue for taking the vitamins, such as placing the vitamin bottle beside the alarm clock that she sets each evening. Likewise, the person who remembers to take their umbrella is the person who sets it beside the door when they hear on the news that it might soon be raining. 8.2 Stimulus Control 321 Stimulus control procedures are also the treatment of choice for sleep-onset insomnia, in which people have difficulty falling asleep. For example, Bootzin, Epstein, and Wood (1991) recommend the following procedure: 1. Go to bed only when you are sleepy. 2. Use the bed only for sleeping (or sex). Do not lie in bed to read, study, or watch television. 3. If you cannot fall asleep within 10 to 20 minutes, get out of bed and go to another room. Go back to bed only when you feel sleepy. 4. Repeat the preceding rule as often as necessary. This rule should also be applied if you are unable to fall asleep after a middle-of-the-night awakening. 5. Use your alarm to get up at the same time each morning, regardless of how you slept the night before. 6. Do not take naps during the day. The obvious goal of the program is to make lying in bed a strong cue for sleeping. Research has shown this to be an effective program, with many people reporting considerable improvement in their sleep habits both immediately following the program and at long-term follow-up (Lichstein & Riedel, 1994). Strong stimulus control is a major characteristic of those behaviors we call habits. A habit is an operant behavior that is so well learned and so well practiced that it seems to automatically occur within a certain setting or when we are exposed to a certain stimulus. It seems to require minimal effort, little or no planning, and may even occur with little or no awareness. Thus, we might automatically tie our shoelaces when we put on our shoes, automatically head for the shower when we get out bed in the morning, and automatically start making coffee when we enter the kitchen. Unfortunately, some of our habits may be counterproductive, such as delving into social media each evening rather than sitting down to study or hitting the snooze button on the alarm each morning rather than getting out of bed. Wouldn’t it be wonderful— as it is for some people—if the opposite were true and we automatically sat down to study each evening and automatically got out of bed each morning? Fortunately, a good deal of information exists, both old and new, on how to go about breaking bad habits and creating good ones (which might also make for a good project if the course you are taking requires one). For more information on this topic, read both the Study Tip insert and the And Furthermore box, “Edwin Guthrie: Stimulus Control for the Practical Person,” in this chapter. behavior management technique called t . 2. Jaclyn’s cat has a terrible habit of jumping up on the kitchen counter whenever she is preparing food. How might Jaclyn use a stimulus control procedure to . eliminate this behavior? 3. A is an operant behavior that is under strong ________________ and seems to au__________________ occur in certain settings. Quick Quiz M 1. Training a rhinoceros to touch the end of a stick with its nose is an example of a 322 Chapter 8 Extinction and Stimulus Control CALVIN AND HOBBES © 1988 Watterson. Dist. By UNIVERSAL UCLICK. Reprinted with permission. All rights reserved Study Tip: Stimulus control is also useful for creating an effective study environment. As we previously noted in Chapter 4, many students study best in an environment in which distractions (cues for behavior other than studying) are kept to a minimum (e.g., Heffernan & Richards, 1981; Plant, Ericsson, Hill, & Asberg, 2005). Students can also benefit from the use of a dedicated study space, such as a particular desk, which is strongly associated with studying. As a result, the mere act of placing oneself in that space can almost automatically evoke the tendency to study (Skinner, 1987; Steel, 2010). Of course, this type of stimulus control cannot be established overnight. Sitting at a desk for three hours trying to study but daydreaming instead will only associate the desk with daydreaming. Better to begin with short high-quality study periods and gradually progress to longer ones. A classic example of such a procedure was reported by Fox (1962). In the initial experiment, the students were instructed to schedule a one-hour study period per day and to spend at least part of that hour studying their most difficult course. They were also told to conduct that studying only in a particular setting, to have only their study materials with them in that setting, and not to be in that setting on other occasions. They were also told that if they became bored or started to daydream, they were to complete just a bit more studying and then leave the setting. Finally, any studying done outside the special one-hour period had to be done elsewhere. Initially, none of the students could study throughout the one-hour period, but they gradually built up the ability to do so. A similar procedure was carried out for each of their other courses, with the result that students were eventually studying each course for one hour per day. This program—which also involved instruction in other academic skills—resulted in considerable improvement in students’ grades. In a second experiment, graduate students who were only infrequently practicing a required foreign language were provided a special setting in which to practice. They were also told to practice each day in that setting for at least 10 minutes, but for no more than one hour. This procedure, which combines stimulus control with the just-get-started tactic discussed in Chapter 7, proved to be highly effective, with all of the students soon engaging in high-quality practice for the full hour each day. Although this study leaves much to be desired from a scientific perspective (what are some of the ways in which it is only a weak demonstration of the effectiveness of these procedures?), you may find it interesting to experiment with using similar procedures to enhance your own studying. (Refer also to “Edwin Guthrie: Stimulus Control for the Practical Person” in the And Furthermore box for an additional study tip.) 8.2 Stimulus Control 323 And Furthermore Edwin Guthrie: Stimulus Control for the Practical Person Edwin Guthrie (1886–1959) was a famous learning theorist who strongly emphasized the role of stimulus control. This is because, from his perspective, all learning is a function of one basic principle: if a behavior occurs in the presence of certain stimuli, then the presence of those stimuli will automatically evoke the behavior (Guthrie, 1952). In other words, Guthrie’s theory is an extreme version of an S–R theory. Guthrie’s theory, which is known as contiguity theory, makes a startlingly blunt prediction about behavior. Whatever you did the last time you were in a certain setting is exactly what you will do the next time you are in that setting. Suppose, for example, that the last time you walked down a certain hallway, you entered the first doorway to the right. Guthrie’s theory predicts that the next time you walk down that hallway, you will again enter the first doorway to the right, given that all the stimuli are the same as when you last walked down that hallway. Of course, this last part is the catch. The stimuli that precede a behavior—and this can include both internal and external stimuli— are never exactly the same from one occasion to the next. Instead, they are only more or less similar, with the result that a behavior is only more or less likely to be repeated. Note also that the consequences of the behavior—for example, perhaps you entered the first doorway to the right because it leads to the cafeteria where you bought coffee—do not enter into the equation. Guthrie viewed consequences as having only an indirect effect on behavior, though his explanation for how this works is too complex to delve into here. Guthrie himself did little research, and the research that was done provided only equivocal support for his theory. As a result, it receives little attention from modern-day researchers. Nonetheless, Guthrie’s approach is still considered a major theory of learning (e.g., Olson & Hergenhahn, 2013). Perhaps one reason for its enduring attraction is the simplicity of the theory (scientists often find parsimonious explanations quite attractive). Another reason is the engaging practicality of the theory. One of Guthrie’s most cited examples is that of a young girl who each day threw her coat on the floor when she arrived home and was each day scolded by her mother (Guthrie, 1952). On the surface, we might speculate that the girl repeated the behavior because it was reinforced by the attention she received from her mother. From Guthrie’s perspective, however, the mother’s reaction had little effect on the behavior. Rather, the stimuli that the girl encountered when she entered the house had become so strongly connected to the response of throwing the coat on the floor that this response automatically occurred each time she entered. To solve the problem, the mother began to insist that the child pick up her coat, go back outside, and then practice the behavior of entering the house and hanging up the coat. The stimuli present when the girl entered the house then became associated with the act of hanging up the coat rather than throwing it on the floor, and the new behavior supplanted the old. Another example is that of a student who was having difficulty studying because she was continually distracted by the sound of a neighbor’s radio (Guthrie, 1952). Instead of trying to force herself to study, the student read mystery stories. The stories were so interesting that she was able to read them without being distracted by the sound of the radio. Within a week, the behavior of concentrating while reading had become so firmly 324 Chapter 8 Extinction and Stimulus Control attached to that setting that she was able to switch back to her study materials and concentrate well despite the radio. This last example implies that students who have difficulty concentrating might sometimes do well to study something interesting before they study something boring. Starting with interesting material might establish a strong level of concentration that will then carry over to the less interesting material. This, of course, seems to contradict Grandma’s rule—or the Premack principle, if you will—which contends that you should work before you play (as applied to studying, you should start with less interesting material and finish off with more interesting material). Guthrie, by contrast, seems to suggest that, for at least some people, it might sometimes be useful to play before you work. In Review To create a summary of this chapter, start with a paragraph that describes operant extinction and the various side effects associated with it. The next paragraph can describe resistance to extinction and the various factors that predict it. A third paragraph could define and provide examples of stimulus control, with an emphasis on distinctions between SD and S". You could write two paragraphs to describe related concepts, including contrast effects, errorless discrimination training, and other stimulus control procedures that are used for animal training. Another way to review material from this chapter is to create a plan to improve your study habits, using various techniques and suggestions we’ve provided. Think carefully about how the cues in your environment affect your ability to concentrate or feel ready to study. You could also create a plan for improving your sleep habits or another behavior (if your study habits are already excellent!). For Further Exploration Adults have temper tantrums, too! This column provides some examples of adult tantrums and suggestions for how to deal with them. You should be able to apply the terminology you learned in Chapters 6 through 8 to these examples: www.psychologytoday.com/ca/blog/life-after-50/201902 /strategy-deal-adult-temper-tantrums. Are you curious about the effectiveness of sleep training for infants? This article provides good fact checking about different methods and lots of!useful links for more information: www.npr.org/sections/health -shots/2019/07/15/730339536/sleep-training-truths-what-science-can -and-cant-tell-us-about-crying-it-out. Study Questions 1. Define extinction as it applies to operant conditioning. What are six side effects of extinction? 2. What is resistance to extinction? What is the partial reinforcement effect? What two basic intermittent schedules of reinforcement are likely to produce the highest resistance to extinction? Concept Review 3. What are four factors other than the schedule of reinforcement that can affect resistance to extinction? 4. What is spontaneous recovery, and how is it affected by successive sessions of extinction? 5. Define a DRO procedure. Why is a DRO procedure often more effective than a straight extinction procedure in eliminating a behavior? 6. Define stimulus control. What are stimulus generalization and stimulus discrimination as they occur in operant conditioning? Diagram an example of a discrimination training procedure (be sure to include the appropriate abbreviations for each component). 7. What is a generalization gradient? How does the shape of the gradient reflect the degree of generalization? With the use of a properly labeled graph, describe what a peak shift effect consists of. 8. Diagram an example of a multiple schedule that involves the response of lever pressing, FR 20- and VI 30-sec schedules of reinforcement, and the stimuli of tone and light. Be sure to include the appropriate label for each component (SD, etc.). How does a multiple schedule differ from a chained schedule? 9. Define positive contrast, negative contrast, and anticipatory contrast and give an example of each. 10. Describe errorless discrimination training and the two basic aspects of this procedure. What is a major drawback of such training? 11. How might a bird owner use stimulus control to eliminate a parrot’s tendency to squawk for long periods of time? How might a novelist use stimulus control to facilitate the act of writing? Concept Review anticipatory contrast. The process whereby the rate of response varies inversely with an upcoming (“anticipated”) change in the rate of reinforcement. behavioral contrast. A change in the rate of reinforcement on one component of a multiple schedule produces an opposite change in the rate of response on another component. differential reinforcement of other behavior (DRO). Reinforcement of any behavior other than a target behavior that is being extinguished. discrimination training. As applied to operant conditioning, the differential reinforcement of responding in the presence of one stimulus (the SD) and not another. errorless discrimination training. A discrimination training procedure that minimizes the number of errors (i.e., nonreinforced responses to the S") and reduces many of the adverse effects associated with discrimination training. extinction. The nonreinforcement of a previously reinforced response, the result of which is a decrease in the future strength of that response. extinction burst. A temporary increase in the frequency and intensity of responding when extinction is first implemented. fading. The process of gradually altering the intensity of a stimulus. 325 326 Chapter 8 Extinction and Stimulus Control generalization gradient. A measure of the strength of responding in the presence of stimuli that are similar to the SD and vary along a continuum. multiple schedule. A complex schedule consisting of two or more independent schedules presented in sequence, each resulting in reinforcement and each having a distinctive SD. negative contrast effect. An increase in the rate of reinforcement on one component of a multiple schedule produces a decrease in the rate of response on the other component. partial reinforcement effect. The process whereby behavior that has been maintained on an intermittent (partial) schedule of reinforcement extinguishes more slowly than behavior that has been maintained on a continuous schedule. peak shift effect. Following discrimination training, the peak of a generalization gradient will shift from the SD to a stimulus that is further removed from the S". positive contrast effect. A decrease in rate of reinforcement on one component of a multiple schedule produces an increase in the rate of response on the other component. resistance to extinction. The extent to which responding persists after an extinction procedure has been implemented. resurgence. The reappearance during extinction of other behaviors that had previously (sometime in the past) been effective in obtaining reinforcement. spontaneous recovery. The reappearance of an extinguished response, despite the continued absence of reinforcement, following a rest period after extinction. stimulus control. A situation in which the presence of a discriminative stimulus reliably affects the probability of a behavior. stimulus discrimination. In operant conditioning, the tendency for an operant response to be emitted more in the presence of one stimulus than another. stimulus generalization. In operant conditioning, the tendency for an operant response to be emitted in the presence of a stimulus that is similar to an SD. Chapter Test 16. When Asha’s parents won the lottery and bought her lots of neat playthings, she became (less/more) interested in school. This is an example of a(n) contrast effect. 4. When Erin was babysitting Lucie, it took hours before Lucie would stop pestering her for a treat (Erin had been instructed to stop giving her treats). The next time Erin babysits Lucie, Lucie will (probably/probably not) resume asking for a treat. This can be considered an example of an extinction effect known as . This may be occurring in this case because the entry of a babysitter into the house is, for Lucie, a(n) stimulus indicating that a treat will soon become available. Chapter Test 10. Lucie is ecstatic when Tamsen is her babysitter and completely indifferent when Natasha is her babysitter. This is because Tamsen tends to give her treats, but Natasha does not. Thus, Tamsen is a(n) (give the abbreviation) for the availability of treats, while Natasha is a(n) ____________________. 27. More persistent is to less persistent as (high/low) resistance to extinction is to ____________________ resistance to extinction. 15. When Trish’s friend Laura spread some nasty rumors about her, Trish stopped talking to her. Laura did not understand the reason for Trish’s silence and initially (increased/decreased) the frequency with which she attempted to talk to Laura. From the perspective of this being an extinction process, Laura’s behavior can be seen as an example of a(n) ____________________. 9. Right after Gina was stung by a hornet, she was as likely to run away from houseflies as from hornets, which is an example of stimulus . One year later, we find that Gina runs away from hornets but not houseflies, which is an example of stimulus ____________________. 19. Leon is a teacher who finds that the children in his class are extremely unruly. To solve this problem, he announces that whenever he is holding up a flag, the children can run around and do whatever they want. Then, periodically throughout the day, he holds up the flag for a few minutes and lets the children run around like crazy. He also finds that when the flag is not being held up, the children are now relatively (quiet/noisy), insofar as the behavior of running around is now under ____________________. 5. When Charles took his daughter, Lucie, to the store, it took hours before Lucie would stop making a fuss and pestering him for a treat. Charles could likely have speeded up this process through the use of a(n) (give the abbreviation) ____________________ procedure. 11. When Mehgan lived in Vancouver, she dated Mike, who was quite uneducated, and David, who was moderately educated. She had a boring time with Mike and a great time with David. She then moved to Dallas and set her sights on meeting someone new. According to the effect, we would expect her to be most interested in meeting someone (as educated as/more educated than) David. 21. In behavioral , a change in the rate of on one component of a multiple schedule is followed by a(n) (similar/opposite) change in the rate of on the other component. 7. On a generalization gradient, the strongest response typically occurs to the . 13. The nonreinforcement of a previously reinforced response defines the of extinction, while the resultant decrease in the strength of that response defines the of extinction. 327 328 Chapter 8 Extinction and Stimulus Control 28. Without even thinking about it, Shahram immediately places his car keys in the dish beside the door when he comes home each evening. This behavior appears to be a(n) that is under strong . 3. A dog whose begging for food has been reinforced 200 times is likely to show greater to extinction than a dog whose begging has been reinforced only 10 times. 20. When Trish’s friend Laura spread some nasty rumors about her, Trish stopped talking to her. Laura tried different ways to get Trish to talk to her: phoning her, e-mailing, writing letters, and sending messages through mutual friends. The many ways in which Laura attempted to interact with Trish are indicative of an effect that often accompanies extinction, which is an increase in the v of a behavior. 25. While teaching his daughter the letters of the alphabet, Vern would say each letter as he showed it to her and encourage her to repeat what he said. He then began to say the letters more and more softly, with the result that she eventually said them on her own without any prompt from him. This can be seen as an example of discrimination learning. One problem with this type of method is that the learning that results from this procedure tends to be (inflexible/too flexible). 22. When Trish’s friend Laura spread some nasty rumors about her, Trish stopped talking to her. Laura tried very hard to get Trish to talk to her. She also became emotionally (upset/distant), which included becoming with Trish. 2. When visiting a tropical resort last summer, you frequently encountered a group of children in the street who were trying to sell souvenirs. Although you always rejected their sales pitches, they were incredibly persistent. Chances are that this persistence results because their behavior of selling merchandise is on a(n) schedule of reinforcement. Another factor would be that the children seemed quite poor; hence, they were relatively of the soughtafter reinforcer. 8. A indicates the strength of responding in the presence of stimuli that are similar to the and that vary along a . 23. When Tamara first moved to the city, she went out each evening and had a great time. One evening at a nightclub, however, she had a frightening experience that really turned her off the club scene. Interestingly, she subsequently became (more/less) interested in other activities, including her job. This may be an example of a(n) contrast effect. 14. The first step in carrying out an extinction procedure is to identify the that is maintaining the behavior. 18. When Trish’s friend Laura spread some nasty rumors about her, Trish stopped talking to her. Laura tried to get Trish to talk to her but quickly Answers to Chapter Test gave up. Laura’s behavior of trying to interact with Trish seems to have (low/high) resistance to extinction. 24. Ahmed found school only slightly interesting. Unfortunately, his lack of studying led to some very poor marks one semester, with the result that he faced the threat of being forced to withdraw for a year. Based on what you have learned in this chapter, throughout the rest of the semester, Ahmed probably became (more/less) interested in his academic studies. This can be seen as an example of a(n) contrast effect. 12. A multiple schedule consists of two or more independent schedules presented (simultaneously/sequentially), each resulting in a(n) ___________________ and each having a distinctive ____________________. 1. When Trish’s friend Laura spread some nasty rumors about her, Trish stopped talking to her. Laura tried hard to get Trish to talk to her. She even asked Trish if she would like to go to the local mall, which had been one of their favorite activities when they first became friends. This may be an example of an extinction effect known as . 17. Yan lives in a very crowded city, so he trains his little boy to stay in close contact with his right hand whenever they are walking in a crowd. This is similar to a behavior management technique known as that is used to guide animals. 6. When the commander yells “Charge!” all of his troops climb out of the trench and start running toward the enemy. The behavior of these troops is obviously under strong control. 26. When Trish’s best friend Laura spread some nasty rumors about her, Trish stopped talking to her. Laura tried very hard to get Trish to talk to her. When Trish refused, Laura eventually became , one symptom of which was a relatively (low/high) level of activity. Answers to Chapter Test 1. resurgence 2. intermittent (or VR); deprived 3. resistance 4. probably; spontaneous recovery; discriminative 5. DRO 6. stimulus 7. SD 8. generalization gradient; SD; continuum 9. generalization; discrimination 10. SD; S" 11. peak shift; more educated than 12. sequentially; reinforcer; SD 13. procedure; process 14. reinforcer 15. increased; extinction burst 16. less; negative 17. targeting 18. low 19. quiet; stimulus control 20. variability 21. contrast; reinforcement; opposite; response 22. upset; angry (or aggressive) 23. more; positive 24. more; anticipatory 25. errorless; inflexible 26. depressed; low 27. high; low 28. habit; stimulus control 329 Chapter 9 Escape, Avoidance, and!Punishment James told Mischa that he was once married to a woman who had been diagnosed with depression. He explained that she had stopped working, moped around the house all day, and would often break down and start crying. Despite his best efforts to be supportive, they finally got a divorce. Mischa felt a lot of sympathy for James, who was obviously a very caring fellow. After several months, though, Mischa noticed that she herself was becoming depressed. Although James was often quite affectionate, he would also become angry or, worse yet, grow coldly silent for no apparent reason. James also had a tendency to contradict Mischa whenever she offered her opinion on some matter, and took special pains to point out her mistakes (because, James said, he loved Mischa so much that he wanted to be honest with her). Mischa eventually learned that James’s former wife had made a remarkable recovery soon after their divorce. Chapter Outline 9.1 Escape and Avoidance Two-Process Theory of Avoidance Avoidance Conditioning and Phobias Avoidance Conditioning and Obsessive–Compulsive Disorder 9.2 Punishment Types of Punishment Problems with the Use of Punishment Benefits and the Effective Use of Punishment Theories of Punishment 9.3 Noncontingent Punishment Learned Helplessness Masserman’s Experimental Neurosis Learning Objectives 9-1 Define and distinguish between escape and avoidance. 9-2 Describe research results about the role of avoidance conditioning for phobias. 330 9.1 Escape and Avoidance 9-3 Describe research results about the role of avoidance conditioning for obsessive–compulsive disorder. 9-4 Define and distinguish among different types of punishment. 9-5 Describe risks associated with the use of punishment. 9-6 Describe the possible benefits of punishment and how it can be effectively used. 9-7 Compare and contrast the different theories of punishment. 9-8 Describe effects of noncontingent punishment, including learned helplessness and experimental neurosis. This chapter explores the effects of aversive consequences on behavior. We begin by examining the role of negative reinforcement in the development of escape and avoidance behaviors. As you will read, this process plays a critical role in the development and maintenance of phobic and obsessive–compulsive disorders in humans. We follow this with a discussion of punishment, in which the presentation or withdrawal of consequences serves to suppress a behavior. We discuss some of the undesirable side effects of punishment as well as some of the ways in which punishment can be effective. The chapter concludes with a discussion of the harmful effects of noncontingent punishment, in which the punishing stimulus is delivered independently of the individual’s behavior. 9.1 Escape and Avoidance As you will recall from Chapter 6, negative reinforcement consists of the removal of an aversive stimulus following a response, which leads to an increase in the future strength of that response. For example, if we wave our hands at a bothersome wasp and the wasp flies away, we will likely repeat that action with the next wasp that annoys us. Negative reinforcement is associated with two types of behavior: (1) escape behavior, in which performance of the behavior terminates the aversive stimulus, and (2) avoidance behavior, in which performance of the behavior prevents the aversive stimulus from occurring. Thus, we escape from the rain when we run indoors after it has started; we avoid the rain when we head indoors before it has started. Typically, one first learns to escape from an aversive stimulus and then to avoid it. This process can be demonstrated using a shuttle avoidance procedure, in which an animal has to shuttle back and forth in a box to avoid an aversive stimulus. In a typical procedure, a rat is placed in a chamber divided by a low barrier. A stimulus of some sort, such as a light, is presented for, say, 10 seconds, followed by a mild electric shock. The rat can escape the shock by climbing over the barrier to the other side of the compartment, as it will quickly learn to do whenever it feels a shock (refer to the top panel of Figure 9.1). Technically speaking, at this early point in the process, the presence of a shock is a discriminative stimulus that sets the occasion for (or evokes) the escape behavior of crossing the barrier. Crossing the barrier is then negatively reinforced by the removal of shock: Shock (SD): Cross barrier (R) ! Removal of shock (SR) 331 332 Chapter 9 Escape, Avoidance, and Punishment Escape and avoidance in a shuttle avoidance task. As shown in the top panel, the animal first learns to escape from the shock by climbing over the barrier whenever a shock occurs. Later, as it learns that the light predicts the occurrence of shock, it climbs over the barrier whenever the light appears, thereby avoiding the shock (as shown in the bottom panel). (Source: Nairne, 2000.) Figure 9.1 Escape Avoidance Now, remember that the shock is preceded by the presentation of a light, which is essentially a warning signal that a shock is about to occur. As the rat learns to associate the light with the shock, it will begin crossing the barrier whenever the light is presented and before the shock begins (refer to the bottom panel of Figure 9.1). The light is now a discriminative stimulus for the avoidance response of crossing the barrier. Light (SD): Cross barrier (R) ! Avoidance of shock (SR) Quick Quiz A In similar fashion, we might first learn to escape from an upsetting conversation with a racist acquaintance by inventing an excuse for leaving. After a few experiences, however, we might begin actively avoiding any interactions with that individual. By doing so, we avoid having to endure any exposure to that person’s racist views. 1. Behavior that terminates an aversive stimulus is called behavior, whereas behavior that prevents an aversive stimulus from occurring is behavior. called 2. Typically, one first learns to from an aversive stimulus it. and then to 3. Julio initially takes vitamin C whenever he has a cold in the hope that it will shorten the duration of his symptoms. Feeling that this is effective, he begins 9.1 Escape and Avoidance taking it daily in the hope that it will keep him from getting a cold. Julio initially took the vitamin C to (avoid/escape) the symptoms of a cold; he later took it to the symptoms of a cold. 4. In the shuttle avoidance procedure described previously, the rat first learns to from the shock, with the acting the as the SD for the behavior. The rat later learns to acting as the SD for the behavior. shock, with the Two-Process Theory of Avoidance Researchers have generally shown more interest in studying avoidance behavior than escape behavior. This is because escape behavior is relatively easy to understand from a theoretical perspective. For example, when escaping from shock by climbing over a barrier, the rat moves from a clearly aversive situation to a nonaversive situation. But the motivation underlying avoidance behavior is less apparent. When climbing over a barrier to avoid shock, the rat seems to be moving from one nonaversive situation (no shock) to another nonaversive situation (no shock). How can a lack of change function as a reinforcer? An early attempt to explain avoidance behavior was the two-process theory of avoidance (also known as the two-factor theory of avoidance) proposed by Mowrer (1947, 1960). According to this theory, two processes are involved in learning an avoidance response. The first process is classical conditioning of a fear response to a CS. For example, in the shuttle avoidance procedure described previously, the light that precedes the shock becomes a CS that elicits a conditioned fear response: Light (NS): Shock (US) ! Fear (UR) Light (CS) ! Fear (CR) Once this conditioned fear has been established, it forms the basis of an operant conditioning procedure. If the CS generates a conditioned fear response, then moving away from the CS should result in a reduction of fear. This reduction of fear should in turn serve as a negative reinforcer for the response that produced it. In our experimental example, presentation of the light elicits a conditioned fear response, while climbing over the barrier produces a reduction in fear that serves as a negative reinforcer for that response. Light (SD): Climb over barrier (R) ! Reduction in fear (SR) Thus, Mowrer’s two-process theory of avoidance proposes that avoidance behavior is the result of two distinct processes: (1) classical conditioning, in which a fear response comes to be elicited by a CS; and (2) operant conditioning, in which moving away from the CS is negatively reinforced by a reduction in fear. 333 Quick Quiz B 334 Chapter 9 Escape, Avoidance, and Punishment 1. It is relatively easy to understand the process underlying (escape/avoidance) conditioning because the organism moves from a(n) situation. By contrast, it is more difficult situation to a non conditioning because the organism moves to understand situation to another from a(n) situation. 2. According to Mowrer, avoidance is the result of two distinct processes: (1) conditioning of a response, conditioning, in which an avoidance response and (2) r by a reduction in . is n Mowrer’s two-process theory generated an enormous amount of research, with researchers soon discovering several difficulties with it. One problem was that avoidance responses are often extremely persistent. Solomon, Kamin, and Wynne (1953), for example, found that dogs would continue to jump a barrier to avoid shock for hundreds of trials even though the shock apparatus had been disconnected and avoidance was no longer necessary. One dog, for example, made more than 600 avoidance responses before the experimenters finally gave up and put a stop to the session. On the surface, it seems as though two-process theory cannot account for such persistence. If the animal repeatedly encounters the CS in the absence of the US, then fear of the CS should eventually extinguish—meaning that the animal should eventually stop jumping over the barrier. But it seemed as though the avoidance behavior would not extinguish. Why not? A possible answer to this question is provided by a modification of two-process theory known as the anxiety conservation hypothesis (Solomon & Wynne, 1954). According to this supposition, avoidance responses usually occur so quickly that there is insufficient exposure to the CS for the conditioned fear to fully extinguish—that is, a good deal of the conditioned fear is conserved because exposures to the CS are too brief for extinction to take place.1 For this reason, avoidance responses can be extremely persistent. In addition, supporters of two-process theory have pointed out that avoidance responses are not as persistent as sometimes claimed (Levis, 1989). If one continues to expose the animal to the aversive CS, extinction will often eventually occur given that there are no further pairings of the CS with the US. Thus, the fact that avoidance responses are extremely persistent might not be as damaging a criticism of two-process theory as was first assumed. Researchers, however, also discovered a second, more serious difficulty with two-process theory. They found that, after repeated avoidance trials, animals appeared to show no evidence of fear but continued to make the avoidance response anyway (Solomon & Wynne, 1953). In other words, once the animals had become adept at making the avoidance response, they seemed to become almost nonchalant and relaxed while carrying it out. This constituted a major 1 According to Eysenck’s (1968) theory of incubation (discussed in Chapter 5), such brief exposures might also strengthen a conditioned fear response, which would further counteract the process of extinction. 9.1 Escape and Avoidance 335 problem for two-process theory: If the animals are no longer afraid of the CS, how can avoidance of the CS be negatively reinforced by a reduction in fear? This was a pretty damaging criticism, and for a while it looked as though two-process theory was nearly refuted. Levis (1989), however, has argued that although animals in avoidance experiments may become significantly less fearful with experience, there is no evidence that they become completely nonfearful. In fact, evidence suggests that if an animal completely loses its fear of the aversive CS, then, just as two-process theory predicts, the avoidance response ceases to occur. But as long as even a slight fear remains, the avoidance response continues, suggesting that fear reduction is still functioning as a negative reinforcer for the behavior (Levis & Boyd, 1979). Other theories have also been proposed to account for avoidance behavior. According to one-process theory, for example, the act of avoidance is negatively reinforced simply by the lower rate of aversive stimulation with which it is associated (Herrnstein, 1969; Herrnstein & Hineline, 1966). Thus, the rat in a shuttle avoidance task persistently climbs over the barrier when the light comes on because this action results in a decreased rate of shock, and not because it results in decreased feelings of fear. The attractive aspect of this theory is that it does away with any reference to an internal state of fear, the existence of which has to be inferred. The overall reduction in aversive stimulation that accompanies avoidance is instead regarded as a sufficient explanation for the behavior. (Refer to Domjan, 2015, for an overview of different theories of avoidance.) The debate over which processes underlie avoidance behavior will likely continue for some time. At the very least, avoidance behavior is turning out to be more complicated than researchers originally suspected. Fortunately, the knowledge gained from all this theorizing and research is proving to have practical application, particularly in the analysis and treatment of anxiety disorders, a topic to which we turn next. . c hypothesis, avoidance responses usually occur so (quickly/slowly) that exposures to take place. to the (CS/US) are too (long/brief) for 3. A second problem with Mowrer’s theory is that after sufficient experience with , avoiding the aversive CS, the animals no longer show any yet they continue to make the avoidance response. Levis, however, contends that such animals are nevertheless still (slightly/strongly) fearful, otherwise the avoidance response would extinguish. 4. According to the one-process theory of avoidance, the avoidance response is negatively reinforced by a reduction in overall rate of av . stimulation, as opposed to a reduction in trials, the avoidance response does not seem to e 2. However, according to the anx Avoidance Conditioning and Phobias In Chapter 5, we noted that the basis of many phobias is the development of a classically conditioned fear response, which fails to extinguish because the individual avoids the feared stimulus. At that time, we focused on the classical Quick Quiz C 1. One apparent problem with two-process theory is that, even after hundreds of 336 Chapter 9 Escape, Avoidance, and Punishment Quick Quiz D conditioning aspect of a phobia. Let us now examine the role of avoidance learning in phobic development. As noted, avoidance learning appears to be a fundamental process in the development and maintenance of phobic behavior. This is no doubt one reason for the intense interest researchers have shown in studying avoidance. Indeed, demonstrations of avoidance learning in laboratory rats have often been regarded as applicable to phobic conditioning in humans. But is avoidance conditioning in the laboratory a true analogue of human phobic conditioning? Does a rat avoid shock in a shuttle avoidance procedure in the same manner that a person avoids dogs after being bitten? In fact, some have argued that there are considerable differences between avoidance conditioning in an experimental setting and human phobic conditioning. Mineka (1985), for example, has claimed that there are limitations in applying models of experimental avoidance to human phobias. For example, experimental avoidance in animals seems to condition less readily than phobic avoidance in humans. Experimental avoidance typically requires at least a few pairings of the CS and the US (e.g., light and shock) before avoidance has been reliably established, and even then is usually less than 100% certain. For example, in a shuttle avoidance task, the rat will occasionally be tardy in climbing over the barrier, with the result that it sometimes receives a shock. By contrast, human phobias often require only a single, brief conditioning trial to produce avoidance that is both strong and persistent. It might take only a single dog attack for a person to develop a dog phobia, and they will consistently avoid dogs thereafter. 1. According to Mineka, a limitation of applying experimental models of avoidance to phobias is that experimental avoidance seems to condition (more/less) readily than does phobic avoidance. 2. Experimental avoidance often requires (one/a few) conditioning trial(s), while conditioning trial(s); phobic avoidance usually requires additionally, (experimental/phobic) avoidance will in some cases eventually extinguish. In response to Mineka’s (1985) concerns about the applicability of experimental avoidance conditioning to human phobias, Stampfl (1987) proposed that an adequate experimental analogue of a human phobia would, among other things, require the reliable establishment of a fear response with only a single, brief pairing of the CS and US and the occurrence of avoidance on 100% of trials. Stampfl acknowledged that when using these criteria, the typical avoidance-conditioning procedure is an inadequate analogue of human phobic conditioning. He then devised an experimental procedure that produced avoidance conditioning that met these criteria. Stampfl’s (1987) procedure focuses on the fact that people with phobias typically make the avoidance response early in the chain of events leading up to the feared stimulus. For example, a person with an elevator phobia 9.1 Escape and Avoidance will plan their day well ahead of time so that they will not be faced with any pressure to take an elevator. They may, for example, arrange an appointment with a dentist whose office is on the main floor of an office building. This type of planning is important because not doing so could result in a direct encounter with a phobic stimulus, which in turn could greatly increase the cost involved in avoiding it (such as by having to climb the stairs to get to a dentist’s office on the 23rd floor). Thus, the phobic individual learns to make the avoidance response early in the chain of events so as to minimize the effort of avoiding. The opportunity to make an early avoidance response is typically absent from most experimental avoidance procedures. Stampfl, however, designed an apparatus that provided just such an opportunity. As depicted in Figure!9.2, the apparatus consisted of an alleyway that was 5 feet long and contained a dark compartment at one end. Each rat was first allowed to explore the alleyway at its leisure, during which time it came to strongly prefer the black compartment (rats generally prefer the dark). The rat was then given a foot shock while in the black compartment, at which point it fled to the far end of the alleyway. Three minutes later, a conveyor belt was turned on that began to slowly carry the rat toward the dark compartment. During this first trial, most rats waited until they reached the black sidewall area of the apparatus before running back to the far end. When they did run back to the far end, they broke a photobeam that stopped the conveyor belt for a three-minute period. The conveyor belt started up again, and the procedure was repeated. This initial session lasted two hours. During the second session, the response requirement for stopping the conveyor belt was increased from FR 1 to FR 10 (i.e., the rat had to run back and cross the photobeam 10 times before the conveyor belt would stop). Stampfl (1987) found that the rats soon learned to run back to the safe area immediately after the conveyor belt started up. In other words, rather than waiting until they reached the black sidewalls before running back, they began running back after traveling only a short distance. In this manner, they were able to minimize the effort involved in breaking the photobeam and stopping the belt. Moreover, under these circumstances, the rats completely Illustration of a 5-foot automated alleyway similar to the one used by Stampfl (1987). Figure 9.2 Direction of conveyor belt Start Photocells Conveyor belt Black sidewalls Dark compartment 337 338 Chapter 9 Escape, Avoidance, and Punishment Quick Quiz E avoided entering the black compartment on more than 1,000 consecutive trials, thereby consistently avoiding the aversive CS that was associated with shock. Furthermore, this persistent avoidance response resulted from only a single brief exposure to shock. In summary, Stampfl’s results seem to confirm that a critical factor in the maintenance of phobic behavior is that the avoidance response occurs early in the sequence of events leading up to the phobic stimulus, thereby minimizing the effort involved in avoiding the stimulus. Such early responding greatly reduces the extent to which the avoidance response can be extinguished, because the individual experiences little or no exposure to the aversive CS. In terms of the anxiety conservation hypothesis discussed earlier, exposure to the aversive stimulus is so minimal that the avoidance response is extremely resistant to extinction. It is therefore not surprising that phobic behaviors are often extremely persistent. 1. A critical aspect of Stampfl’s experimental analogue of phobic conditioning is that the avoidance response can occur (early/late) in the sequence of events leading up to the feared CS, thereby (maximizing/minimizing) the amount of effort involved in making the response. 2. This results in (little/considerable) exposure to the feared CS, thereby greatly (increasing/ . reducing) the likelihood that the fear response will e Avoidance Conditioning and Obsessive–Compulsive Disorder Phobia is one type of disorder in which avoidance conditioning plays a critical role. Another is obsessive–compulsive disorder (OCD), a disorder characterized by persistent thoughts, impulses, or images (called obsessions), and repetitive, stereotyped actions (called compulsions) that are carried out in response to the obsessions. For example, a person might have an obsessive worry about contacting germs; this leads to a compulsive tendency to take a shower and clean the house many times each day. Or a person might have an obsessive worry about whether they locked their apartment door when they left that morning, which leads to a compulsive pattern of returning to the apartment several times a day to check it. Note that the person recognizes that the compulsive behavior is clearly excessive but nevertheless feels compelled to perform the action. (Interestingly, cleaning and checking are the two most common forms of compulsive behavior.) OCD was once considered a particularly difficult disorder to treat. This changed when clinicians began analyzing OCD in terms of avoidance conditioning, especially Mowrer’s two-process theory (Rachman & Hodgson, 1980). The applicability of this theory to OCD lies in the fact that obsessions and compulsions have opposite effects on anxiety. In general, obsessional thoughts are associated with an increase in anxiety, whereas compulsive behaviors are associated with a decrease in anxiety. For example, a person who has a contamination fear and is a compulsive cleaner usually experiences an increase in anxiety after exposure to situations in which “contamination” 9.1 Escape and Avoidance might have occurred, such as when taking out the garbage. Garbage elicits such strong anxiety that any part of the body that has touched garbage also elicits anxiety. Taking a shower, however, results in the removal of this anxiety. Thus, from the perspective of two-process theory, the obsessional feeling of anxiety is a classically conditioned response elicited by contact with the garbage, while the compulsive showering is an operant response that is negatively reinforced by a reduction in anxiety. The role of avoidance in OCD is virtually the same as in phobic behavior, except that OCD typically involves an active avoidance response while phobic behavior typically involves a passive avoidance response. That is, a person with OCD will generally do something in order to reduce anxiety (such as showering), while a person with a phobia will generally not do something in order to reduce anxiety (such as not go near a dog). Nevertheless, individuals with OCD can also utilize passive avoidance responses (e.g., by avoiding garbage whenever possible) such that some of their behavior patterns can also be characterized as phobic. Two-process theory helped clarify our understanding of OCD and led to the development of the first effective treatment for the disorder. If a compulsive behavior pattern (such as excessive washing) is maintained by avoidance of an anxiety-arousing event (such as contact with germs), then preventing the avoidance response from occurring should result in the eventual extinction of anxiety. This treatment method is known as exposure and response prevention (ERP), a method of treating OCD that involves prolonged exposure to the anxiety-arousing event while not engaging in the compulsive behavior pattern that reduces the anxiety (e.g., Steketee & Foa, 1985). As with recent versions of exposure-based treatments for phobic behavior, ERP combines the graduated exposure of systematic desensitization with the prolonged exposure of flooding therapy. For example, a compulsive cleaner might be required to first touch objects associated with slight anxiety (such as door handles and hand rails), then objects associated with moderate anxiety (such as garbage cans and dogs), and finally objects associated with intense anxiety (such as dead birds and dog excrement). These graduated exposures are first carried out imaginally—given that the person has good imagery ability—and then in vivo—given that live exposure to the anxietyarousing event is practical. The exposures are also relatively long, often 90 minutes or more, to ensure sufficient time for the anxiety to begin to extinguish. In addition to scheduled treatment sessions, the client is told to practice exposures at home. The client is also told not to perform any compulsive behavior patterns; for example, a compulsive washer might be instructed to avoid all nonessential showers except for one 10-minute shower every five days! Once the obsessive–compulsive pattern has been successfully eliminated, normal patterns of behavior are reestablished (Steketee & Foa,!1985). Although Mowrer’s two-process theory has proven quite useful in enhancing our understanding and treatment of OCD, it does not provide an entirely adequate explanation for it (Steketee & Foa, 1985). For example, 339 340 Chapter 9 Escape, Avoidance, and Punishment Quick Quiz F people with OCD are usually unable to recall any particular conditioning event that could account for the obsessional anxiety response. People who have a contamination fear, for example, typically do not recall, say, falling into a cesspool before the onset of the fear. On the other hand, onset of OCD does often coincide with a period of stress. One possibility, therefore, is that stress sensitizes certain individuals in such a way that normal concerns, such as those about cleanliness and safety, become greatly exaggerated. Thus, just as the process of selective sensitization might lead to the development or strengthening of a phobia (as discussed in Chapter 5), so too it might lead to the development of OCD. Furthermore, just as genetic factors may predispose some people to develop a phobia (also discussed in Chapter 5), so too some people might have a genetic predisposition toward developing OCD (Billet, Richter, & Kennedy, 1998). On a more cognitive level, people with OCD often hold the irrational belief that they should be in complete control of their thoughts—failing to realize that intrusive thoughts are not uncommon and that most people simply ignore them. In other words, they fail to realize that some thoughts are essentially respondents (reflexes) that are automatically elicited by certain stimuli, and that it is futile to try to control such thoughts. People with OCD also have a tendency to feel personally responsible for events that are highly improbable. They therefore carry out various safety actions, such as rechecking doors and stoves, that other people would not bother with (Salkovskis, 1998). Given the involvement of cognitive factors in OCD, attempts have been made to combine ERP with cognitive therapy on the assumption that directly modifying these false belief systems might enhance treatment. The research, however, indicates that, while cognitive interventions may play a supplementary role in treatment, such as by increasing adherence to the treatment procedure, ERP remains the most critical aspect of treatment (McKay et al., 2015). 1. Janice continually worries that her alarm clock might not be set correctly, and 2. 3. 4. 5. that she will wake up late for class. She therefore checks the alarm clock about 20 times each night before finally falling asleep. The persistent thoughts about the alarm clock not being set are classified as a(n) (compulsion/obsession) while . the frequent checking of the clock is classified as a(n) In general, (compulsions/obsessions) are associated with an increase in anxiety, are associated with a decrease in anxiety. whereas From the perspective of two-process theory, this decrease in anxiety likely r for the functions as a n compulsive behavior. Exposure and response prevention (ERP) therapy for OCD involves prolonged exposure to anxiety-arousing events while (engaging/not engaging) in the (obsessive/compulsive) behavior that serves to reduce the anxiety. ERP is similar to systematic desensitization in that exposure to the anxietyprovoking event is usually (gradual/sudden). It is similar to flooding therapy in that exposure to the anxiety-provoking event is (brief/prolonged). 9.1 Escape and Avoidance 341 6. People with OCD are usually (able/unable) to recall a particular conditioning event that was the cause of the obsessional anxiety response. The disorder often . This suggests that arises, however, during times of s may exacerbate normal concerns about a process of s cleanliness and safety. 7. People with OCD fail to realize that intrusive thoughts are (common/uncommon) and that such thoughts are often (controllable/uncontrollable). They also (take/ fail to take) responsibility for highly (probable/improbable) events. And Furthermore Approach-Avoidance Conflict (or “I Can’t Believe I Volunteered for This!”) Given that avoidance behaviors can be so strong and persistent, you might wonder how it is that we sometimes commit ourselves to events that we later wish we had avoided. Why does Jan, for example, regularly agree to meet an old high school acquaintance for dinner when they have completely separate lives and almost nothing to talk about (though Jan’s acquaintance seems blissfully unaware of it). These situations typically involve what is known as an approach-avoidance conflict, involving events that are in some ways rewarding and in some ways punishing, that is, the events have both appetitive and aversive aspects to them (Dollard & Miller, 1950). The critical factor governing this conflict is the manner in which distance affects the strength of these different consequences. If you recall from Chapter 6, immediacy has a strong effect on the value of a reward, with immediate rewards typically being much stronger than delayed rewards. As it turns out, immediacy can have an even more powerful effect on punishers, with immediate punishers being far stronger than delayed punishers. What this means is that when an event is still some distance away (in either time or space), the rewarding aspects of the event may outweigh the aversive aspects of the event. For this reason, agreeing to get up early on a Saturday morning to help a friend of a friend shingle their house can seem like a great idea when the event is still a week away. After all, you typically enjoy helping others and need the exercise anyway. As time passes, however, the aversive aspects of the event may increase to the point of becoming considerably stronger than the rewarding aspects. Thus, on Friday evening, you can’t believe you ever made such a foolish promise; the aversive aspects of dragging yourself out of bed to do hours of free labor for a virtual stranger now greatly outweigh any rewards associated with it. This pattern of behavior has been demonstrated in rats (refer to Dollard & Miller, 1950). They are first trained to run down an alleyway to obtain food in a goal box. On one trial, however, they also receive a shock in the goal box; as a result, on the next trial, they will very likely run partway to the goal and stop. In the start box, the possibility of obtaining food outweighs the possibility of a shock and the rats are motivated to run forward; but as they approach the goal box, the possibility of receiving a shock now becomes dominant. In other words, far from the goal, the appetitive aspects outweigh the aversive aspects; near the goal, the aversive aspects outweigh the appetitive aspects. 342 Chapter 9 Escape, Avoidance, and Punishment This process may also account for why some couples repeatedly break up and get back together again. They originally broke up because the aversive aspects of the relationship were beginning to outweigh the rewarding aspects. After breaking up, however—in which case they are now more “distant” from each other—the rewarding aspects of the relationship come to the fore. They think more often about the good times they had together and begin to lament ever having broken up. But if they get back together again, the aversive aspects now become salient, and they break up again. Some couples can spend years vacillating back and forth through this cycle. Fortunately, the extent to which a relationship is rewarding versus punishing is, for various other reasons, always shifting up or down, and consequently usually resolves itself one way or the other. 9.2 Punishment Escape and avoidance conditioning involves the strengthening (i.e., reinforcement) of a behavior through the removal of an aversive stimulus. By contrast, punishment involves the weakening of a behavior through the application of an aversive stimulus or the removal of an appetitive stimulus. In this section, we discuss various types of punishment, as well various concerns about the application of punishment. We also briefly describe various theories of punishment. Types of Punishment Let us begin by reviewing the basic distinction between positive and negative punishment. Positive punishment consists of the presentation of a certain event following a response, which leads to a decrease in the future strength of that response. In simple everyday terms, the behavior results in the delivery of something the person or animal hates, so the subject is less likely to behave that way in the future. Being scolded for swearing and being reprimanded for talking back to the boss are both examples of positive punishment (given that these consequences result in a subsequent decrease in the frequency of these behaviors). By contrast, negative punishment consists of the removal of a certain event following a response, which leads to a decrease in the future strength of that response. In everyday terms, the behavior results in the removal of something the person or animal likes, so the subject is less likely to behave that way in the future. A loss of employment for being obnoxious and a loss of dessert for complaining at the dinner table are both examples of negative punishment (again, given that the consequence results in a subsequent decrease in such behavior). Note that the events being removed are pleasant events that can typically serve as positive reinforcers; thus, negative punishment can also be defined as the loss of a positive reinforcer (a pleasant event) following a response. There are two basic types of negative punishment: time-out and response cost. Time-out involves the loss of access to positive reinforcers for a brief period of time following the occurrence of a problem behavior. Time-out has become 9.2 Punishment www.CartoonStock.com popular with many parents today, who frequently attempt to punish a child’s misbehavior by sending them to the bedroom or making them sit in a corner for several minutes. Unfortunately, time-out procedures are often poorly applied and thus have little effect on the problem behavior. For example, it is likely to be ineffective if the time-out setting is actually more reinforcing than the setting from which the child was removed. In fact, sending a child to their room for acting out at the dinner table might reinforce rather than punish the behavior of acting out if the child dislikes sitting at the dinner table. Another problem is the time-outs are often too long. The purpose of time-out is not to get the child “out of your hair” for a period of time but to facilitate the development of more appropriate behaviors. Those appropriate behaviors need to be reinforced, which cannot be done if the child is sitting in their room for hours on end. Time-outs should be brief, especially for young children. In fact, in some cases, a time-out that lasts a minute may be all that is required to effectively decrease the future occurrence of the unwanted behavior, especially if one immediately sets out to reinforce more appropriate behaviors as soon as the child is returned to the normal setting (Miltenberger, 2012). 343 344 Chapter 9 Escape, Avoidance, and Punishment Quick Quiz G The other type of negative punishment is response cost, which is the removal of a specific reinforcer following the occurrence of a problem behavior. Receiving a fine (which involves a loss of money) for speeding or taking a child’s toys away because the child was too noisy are examples of response cost. An advantage of a response cost procedure is that one can easily adjust the severity of the punishment to suit the behavior being punished. Slight aggression with a younger sibling could result in the loss of dessert, while more severe aggression could result in the loss of dessert as well as the opportunity to watch television that evening. A drawback to response cost, however, is that you must clearly identify a reinforcer that, if removed, will have an impact on behavior. It therefore requires a more careful analysis of the situation than a time-out procedure does. (Refer to Miltenberger, 2012, for a more complete discussion of time-out and response cost procedures.) Note that negative punishment is quite different from extinction, even though both involve the removal of reinforcers and both result in a decrease in the strength of a behavior. In the case of extinction, you remove the reinforcer for a behavior, and the person therefore stops performing the behavior. Once Jason’s whining no longer results in cookies, he will eventually stop whining. In the case of negative punishment, however, performing the behavior results in the loss of a reinforcer that the person would otherwise possess. Imagine, for example, that Jason has already received some cookies, or is about to receive some cookies, but then starts whining for a soda pop. If each time he whines, one of his cookies is taken away, he is likely to stop whining. Thus, to distinguish between extinction and negative punishment, ask yourself whether the behavior grows weaker because performing the behavior no longer leads to something (in which case, the process is extinction) or because performing the behavior leads to the removal of something that you would otherwise possess (in which case the process is negative punishment). More generally, just as in Chapter 8, when we noted that a decrease in behavior does not necessarily indicate that extinction is occurring, a decrease in behavior does not necessarily indicate that punishment is occurring. Consider, for example, two children who are fighting over a board game they are playing. If you take away the board game, the fighting will naturally cease. But this by itself does not indicate that taking away the board game punished the act of fighting. All that you have done is simply remove the reason the fighting occurred, and there is now no longer a game for them fight over. Whether or not taking the game away also constitutes an act of punishment (negative punishment, in this case) depends on whether taking the game away when they fight over it also reduces the likelihood of fighting the next time they play that game. 1. When the cat sat at your feet and meowed annoyingly during breakfast one morning, you sprayed it with water. As a result, the cat did not come near the!table or meow the next time you sat down for a meal. The consequence for the cat’s meowing consisted of the (presentation/removal) of a stimulus, and the cat’s behavior subsequently (decreased/increased) in frequency. Therefore, this is . an example of 9.2 Punishment 2. Negative punishment involves the (presentation/removal) of a stimulus following 3. 4. 5. 6. 7. 8. 9. a response that subsequently results in a(n) (increase/decrease) in the future likelihood of that response. When Bobbi threw a temper tantrum, her mother turned off the television program that Bobbi was watching. Bobbi’s mother was attempting to apply a (response cost/time-out) procedure. When Bobbi threw a temper tantrum, Bobbi’s mother made her sit in the corner for a minute. Bobbi’s mother was attempting to apply a (response cost/time-out) procedure. A(n) (advantage/disadvantage) of a time-out procedure is that one (does/ does not) have to clearly identify a specific reinforcer before implementing the procedure. A(n) (advantage/disadvantage) of a response cost procedure is that one (can/cannot) easily modify the severity of the punishment to suit the behavior. Whenever Van whines for a snack, his brother immediately stops playing with him. As a result, Van soon stops whining for a snack. This is an example of (extinction/negative punishment). Van’s brother used to play with Van whenever he whined for him to do so, but stopped doing so. As a result, Van’s whining soon ceased. This is an example of (extinction/negative punishment). If the frequency of a behavior decreases because performing the behavior no longer results in a treat, the process involved is (extinction/negative punishment). If the frequency of a behavior decreases because each time you perform the behavior, it leads to the removal of a treat that you already have, the process involved is (extinction/negative punishment). Shana is banging her toy on the coffee table. Her mother puts a stop to it by taking away the toy. This is an example of (negative punishment/extinction/ neither of these). Punishment can also be differentiated in other ways. For example, just as one can distinguish between intrinsic and extrinsic reinforcement, one can distinguish between intrinsic and extrinsic punishment. Intrinsic punishment is punishment that is an inherent aspect of the behavior being punished. In other words, the activity itself is punishing, such that the person performing the behavior is now less likely to repeat it. Doing push-ups is intrinsically punishing if you are less likely to do push-ups in the future because of how unpleasant it was. Extrinsic punishment is punishment that is not an inherent aspect of the behavior being punished, but is the result of an event that follows the behavior. This event is what we would call an extrinsic punisher. Being chastised after lighting up a cigarette (“Are you still indulging in that filthy habit?”) is extrinsically punishing if it subsequently reduces how frequently you smoke. As with reinforcers, one can also distinguish between primary and secondary punishers. A primary (or unconditioned) punisher is an event that is innately punishing. Loosely speaking, these are events that we are born to dislike. Electric shock, intense heat, and loud noise are examples of primary punishers. A secondary (or conditioned) punisher is an event that has become 345 346 Chapter 9 Escape, Avoidance, and Punishment punishing because it has in the past been associated with some other punisher. For example, if shock is an effective punisher, then a tone that has been paired with shock in a classical conditioning procedure: Tone (NS): Shock (US) ! Fear (UR) Tone (CS) ! Fear (CR) can become a conditioned aversive stimulus that can then be used as a secondary punisher. For example, presentation of the tone could now be used to punish wheel running: Wheel (SD): Running in wheel (R) ! Tone (SP) Quick Quiz H Human behavior is often under the control of secondary punishers. A traffic ticket can effectively punish our tendency to speed, and an icy stare from our partner might effectively punish our tendency to drink too much at a party. Both the fine and the stare are punishing because they have been associated with other types of aversive events: loss of money in the one case and heated arguments in the other. A special type of secondary punisher is a generalized (or generalized secondary) punisher, which is an event that has become punishing because it has in the past been associated with many other punishers. The icy stare is probably best categorized as a generalized punisher because disapproving looks have no doubt been associated with numerous unpleasant events such as reprimands as a child, relationship arguments as an adult, and disciplinary action during one’s stint in the army. 1. Exercising to the point of exhaustion is for many people likely to be an (extrinsically/intrinsically) punishing event. 2. The taste of spoiled food will likely, for most people, function as a (primary/ secondary) punisher, while a restaurant that regularly serves such food will punisher. function as a 3. Looking at an old photo album reminds you of your loneliness as a child, the loss of a favorite pet, and a childhood friend who died. As a result, you stop looking at punisher. it. The old photo album can be classified as a g Problems with the Use of Punishment Although many people believe that behaviorists promote the use of punishment, behaviorists in fact have a general bias against it. This bias results from several problems that are associated with punishment (e.g., Newsom, Favell, & Rincover, 1983; Van Houten, 1983): 1. Punishment of maladaptive behavior doesn’t directly strengthen the occurrence of adaptive behavior, which is what one typically hopes to achieve. Punishing the act of playing aggressively with other children will not automatically result in the child playing more cooperatively. 9.2 Punishment 2. Punishing one behavior may simply result in a generalized suppression of other behaviors. A child who has been punished for playing aggressively will not necessarily begin playing more cooperatively; they might instead simply stop playing with other children, which is not at all what was intended. 3. The person delivering the punishment could become an SD for punishment, with the result that the unwanted behavior is selectively suppressed only when that person is present. The child, for example, might come to view one parent as a discriminative stimulus for punishment and therefore continue to misbehave when that parent is absent. The child has thus learned not to get caught misbehaving, rather than not to misbehave. 4. Punishment might simply teach the individual to avoid the person who delivered the punishment. A child who is often punished by one parent might begin minimizing the time they spend with that parent. This would obviously be less than ideal, especially because the parent and child would miss out on positive interactions. 5. Punishment is likely to elicit a strong emotional response. This is especially the case with the use of positive punishment, such as spanking or yelling, which is likely to result in crying or other displays of distress. These strong emotional responses are not only unpleasant but will also interfere with any subsequent attempt to teach the child more appropriate behavior. A child who is crying uncontrollably is not in an ideal state for learning anything new, such as how to play appropriately. 6. Punishment can sometimes elicit an aggressive reaction. Earlier in this text, we mentioned how a painful stimulus, such as electric shock, can elicit attack behavior in rats. Humans can also react with anger when subjected to aversive stimulation. This anger can be directed toward the individual responsible for the aversive event or, if there are inhibitions about doing so, can be directed toward a substitute target. Thus, a child who is harshly punished for being noisy might not aggress toward the parent who spanked them but instead aggress toward their younger sibling when the parents have left the room. 7. The use of punishment, through the process of modeling, could teach the person that punishment is an acceptable means of controlling behavior. As a result, children whose behavior is being controlled through the use of punishment may themselves begin punishing others to control the behavior of others. 8. Because punishment often has an immediate effect in stopping an unwanted behavior, the use of punishment is often strongly reinforced. If hitting one’s children has the immediate effect of getting them to stop making noise (an immediate negative reinforcer), then the behavior of hitting them has been strongly reinforced. The use of punishment can therefore be quite seductive, enticing the parent to use it more and more frequently, and becoming abusive. 347 Quick Quiz I 348 Chapter 9 Escape, Avoidance, and Punishment 1. Punishment, especially (positive/negative) punishment, can often elicit a reaction. This reaction might include that, if not directed toward the punisher, might be directed toward a substitute target. 2. Punishment of an inappropriate behavior (will/will not) directly strengthen the occurrence of an appropriate behavior. It might even result in a suppression of behavior. g 3. The use of punishment could, through the process of m , teach the recipient that punishment is an acceptable means for modifying a person’s behavior. Yelling at your dog for chewing your slippers might teach the rather than the slippers. Yelling at your dog to avoid dog for chewing your slippers might also teach your dog not to chew the slippers . only when 4. If punishment has an i effect in getting someone to stop annoying us, this result can act as a strong n for us to use punishment in the future. r strong e Study Tip: Punishment may also play a role in the tendency to procrastinate. For example, Robert Boice (e.g., 1989, 1996) has conducted detailed analyses of the work habits of faculty members, distinguishing those who are productive writers from those who are not (the writing of articles and books arguably being one of the most challenging tasks professors face). A major finding was that productive faculty tended to engage in short, consistent writing sessions spread throughout the week, while nonproductive faculty tended to engage in intermittent “binge” episodes of writing spread far apart—that is, they frequently procrastinated, and when they did write, they wrote intensively for long periods of time. Although the procrastinators believed that long, intense writing sessions were necessary to be productive, Boice concluded that binge writing itself was part of the problem. However invigorated one might feel during an intense session of writing—and procrastinators often reported this—this pattern of work is so effortful that one soon starts to avoid it. In essence, binge writing sessions are sufficiently aversive by the time one finishes the session that they punish the act of writing. This can set up (as discussed earlier in this chapter) an approach-avoidance conflict in which a professor may have the best of intentions to work on the writing each day or week but continually finds last-minute excuses to avoid doing so. Based on these results, Boice devised workshops to help professors overcome their tendency to procrastinate. A major component of these workshops is to learn to write in brief daily sessions, perhaps only 30 minutes per day at first, with the act of stopping on time considered as important as starting on time. The goal is to break the tendency to engage in binge writing sessions and to begin to experience writing as less aversive and more enjoyable. Combined with other tactics, such as establishing a balanced lifestyle, arranging a comfortable (nonaversive) writing environment, and seeking out mentors for feedback, these workshops have proven to be very effective in helping many faculty members become more productive. Although Boice’s advice is directed toward faculty, it has obvious implications for students. Students should seriously consider the benefits of engaging in shorter, more 9.2 Punishment 349 consistent study sessions throughout the semester (though, for most students, a certain amount cramming around deadline dates and exam time is probably unavoidable). Consider too that if you study just one extra hour per weekday in, say, the first eight weeks of the semester—during which time many students do only minimal studying— this will add up to 40 extra hours of studying! This is a whopping amount of studying, especially if it is high-quality studying, and would very likely have a significant impact on your grades. You should also consider systematically breaking up long study sessions into a series of brief sessions with frequent breaks for rest and recuperation.2 (If you haven’t already read it, additional information on overcoming procrastination can be found in the appendix: A Brief Guide to Behavior Self-Management). 2 Another excellent (and very readable) book on how to become a more productive writer is How to Write a Lot: A Practical Guide to Productive Academic Writing by Paul J. Silvia (2018). Silvia is himself a psychologist, and the book is directed specifically toward psychology faculty and students. Silvia does a particularly good job of demolishing the typical excuses for not writing, such as needing to “find time” to write (nice try, but time is not lost and does not need finding; you need to schedule time for writing just like you schedule time for work and for school). The book is also written from an explicitly behavioral perspective, with practical advice on such things as the importance of establishing clear goals, the usefulness of tracking one’s progress, and the benefits of selfreinforcement. Benefits and the Effective Use of Punishment For the reasons outlined previously, most behaviorists are extremely cautious about using punishment. Nevertheless, there may be some circumstances in which punishment is judged appropriate, such as in quickly suppressing a potentially dangerous behavior in a young child (e.g., stopping a child from injuring another child). This is especially the case given that alternative interventions, such as extinction and reinforcement of other behaviors, often take considerable time to have an effect. In addition to quickly suppressing a particular behavior pattern, punishment can also have some beneficial side effects (e.g., Newsom et al., 1983; Van Houten, 1983): 1. Punishment can sometimes lead to an increase in social behavior. For example, a young boy who has been scolded for playing aggressively with his sister may become more affectionate toward his sister when the timeout period has ended. Thus, although punishment does not directly teach more appropriate behaviors, such behaviors may sometimes arise as a side effect. Why would such increases in social behavior occur? One possibility is that punishment may activate an innate tendency in humans to become more sociable in an effort to restore one’s relationship with others. 2. Paradoxically, punishment sometimes results in an improvement in mood, such as less crying. This is the opposite of what one would usually expect, which is that punishment would lead to more emotional behavior, not less. In some cases, however, it may be that the child was misbehaving because 350 Chapter 9 Escape, Avoidance, and Punishment they were in some way agitated; in such cases, the punishment might distract the child and disrupt the agitation. 3. Punishment can increase attention to the environment, as shown by increased eye contact and interest in ongoing activities. Shouting at a child, for example, might motivate the child to become more vigilant. This can be especially valuable with children who tend to pay little attention to what is happening around them. In summary, under some circumstances, the application of punishment may be justified and beneficial. If punishment is used, however, the following requirements should be met to maximize the possibility that it will be effective. 1. As much as possible, punishment should be immediate rather than delayed. Unfortunately, in the real world, delayed punishment is often the rule rather than the exception. A child’s misbehavior is frequently discovered only several minutes or hours after its occurrence, and the delivery of a reprimand following such a long delay may have little effect. This is particularly the case with very young children and animals who, because they are unable to understand explanations, are unlikely to associate the punishment with the unwanted behavior. For example, yelling at a dog for making a mess on the carpet several hours after the incident has occurred will probably only upset the animal and do little to prevent future mishaps. 2. At least at the outset, punishment should consistently follow each occurrence of the unwanted behavior. Punishment tends to be less effective in suppressing a behavior when only some instances of the unwanted behavior are punished. In other words, unlike intermittent reinforcement, which has a strong effect on behavior, intermittent punishment tends to have a weak effect on behavior. Nevertheless, once the behavior has been effectively suppressed through consistent punishment, intermittent punishment may be sufficient to maintain the suppression, especially if nonpunished occurrences of the maladaptive behavior are not being intermittently reinforced (Clark, Rowbury, Baer, & Baer, 1973). 3. Punishment should be intense enough from the outset to suppress the target behavior (though—and this is the tricky part—not so intense as to be unnecessarily abusive). If one begins with a very mild punisher that is ineffective, and gradually increases the intensity, it might require a very intense punisher to eventually suppress the unwanted behavior. For example, Miller (1960) presented rats with a very mild shock whenever they entered an alleyway and gradually increased the intensity of the shock until the rats eventually ceased entering the alleyway. He found that the behavior ceased only with high levels of shock, far beyond what would normally have been required to suppress such a behavior. Likewise, a father who initially uses a very mild reprimand to try to get his daughter to stop teasing the dog and gradually increases the severity of the reprimand might eventually have to deliver a very severe reprimand or worse before she will comply. But if the father had started with a 9.2 Punishment 351 moderately severe reprimand, the daughter might have immediately complied, thereby saving the two of them (as well as the dog) a lot of grief. By starting with such a mild intervention and gradually increasing its severity, the father essentially allowed the daughter to adapt to the punishing stimulus. 4. Negative punishment is generally preferable to positive punishment. Negative punishment procedures, such as time-out and response cost, are generally less likely to produce many of the harmful side effects associated with punishment as opposed to positive punishment procedures such as spanking and yelling. Negative punishment can, nevertheless, become abusive, such as when children are forced to endure extremely long time-out periods or are exposed to severe response cost procedures, such as removal of sufficient food. 5. With individuals who have language capacity, punishment is more effective when accompanied by an explanation. A possible reason for this is that an explanation will help clarify the exact behavior that is being punished, thereby making it easier for the child to avoid punishment in the future. This accords with the more general recommendation that children should be given frequent feedback about their behavior, both good and bad, because this will greatly facilitate the children’s attempts to learn appropriate behavior (Craig, Kermis, & Digdon, 1998). 6. Punishment of inappropriate behavior should be combined with positive reinforcement for appropriate behavior. This is perhaps the most important rule. As with extinction, punishment of unwanted behavior will be more effective if it is combined with differential reinforcement for other behavior, especially behavior that is incompatible with the target behavior. This requires that we think carefully about which behaviors we want to increase, not just which behaviors we want to decrease. As the appropriate behavior is strengthened, it will come to supplant the inappropriate behavior. Thus, applying a time-out period to a child for playing inappropriately might have little effect if the child’s behavior of playing appropriately has not been adequately reinforced. Time-out periods should, therefore, be followed by abundant reinforcement for appropriate behavior. In fact, differential reinforcement of appropriate behavior (which might include functional communication training as discussed in Chapter 8) should always be considered the primary tool for eliminating unwanted behaviors. (For a complete discussion of issues involved in the punishment of human behavior, refer to Axelrod & Apsche, 1983.) behavior, improvements in m_______________, and increased a_________________ to the environment. 2. With verbally proficient humans, punishment tends to be more effective when it is accompanied by an e____________________. 3. In general, when implementing a punishment procedure, one should begin with a punisher of sufficient i____________________ to suppress the behavior. Quick Quiz J 1. Beneficial side effects of punishment include increases in s____________________ 352 Chapter 9 Escape, Avoidance, and Punishment 4. Unlike reinforcement, punishment tends to have a stronger impact on behavior if delivered (consistently/intermittently). 5. In general, when attempting to punish a maladaptive behavior, one should also attempt to more adaptive behavior. 6. If punishment is to be used, it should be im , since punishment tends to be relatively ineffective. d 7. In general, (positive/negative) punishment is preferred insofar as it is likely to have (more/fewer) side effects. Theories of Punishment Although a good deal of research has gone into investigating the effectiveness of punishment, less attention has been paid to developing and testing various theories of punishment. We will nevertheless briefly consider three theoretical approaches to punishment. Conditioned Suppression Theory This theory is based on early work by Skinner (1938). He found that although punishment can quickly suppress a behavior, the behavior often quickly recovers when the punishment is withdrawn. What Skinner (1953) assumed was happening was that punishment generates an emotional response that tends to suppress any ongoing appetitive behavior. Crudely put, when the rat is shocked for pressing a lever that produces food, it becomes so upset that it loses interest in the food and therefore stops pressing the lever to obtain it. If, however, the shock is withdrawn, the rat resumes lever pressing as soon as it calms down enough for its interest in food to be revived. By analogy, if Tyler no longer teases his sister after being scolded for doing so, it is simply because he is too upset to pay much attention to his sister. Thus, the conditioned suppression theory of punishment assumes that punishment does not weaken a behavior but instead produces an emotional response that interferes with the occurrence of the behavior. The temporary effect that Skinner (1938) found when he attempted to punish a rat’s behavior led him to conclude that punishment is an ineffective means for producing a lasting change in behavior. Skinner’s experiment, however, utilized a relatively weak form of punishment: a device that slapped the rat on the paw when it attempted to press a lever. Subsequent research revealed that more intense forms of punishment, such as strong electric shocks, are capable of suppressing behavior for much longer periods of time (Azrin & Holz, 1966). Avoidance Theory of Punishment According to the avoidance theory of punishment, punishment actually involves a type of avoidance conditioning in which the avoidance response consists of any behavior other than the behavior being punished (e.g., Dinsmoor, 1954). In other words, just as the behavior of jumping over a barrier is strengthened by shock avoidance in a shuttle avoidance situation, so too is the behavior of doing “anything other than lever 9.2 Punishment pressing” reinforced by shock avoidance in a punishment-of-lever-pressing situation. Thus, in carrying out the following punishment procedure: Lever press (R) ! Shock (SP) one is actually carrying out the following avoidance conditioning procedure: Any behavior other than lever pressing (R) ! No shock (SR) in which any behavior other than lever pressing is negatively reinforced by the absence of shock (e.g., Dinsmoor, 1954). Therefore, according to this theory, Tyler no longer teases his sister after being scolded for doing so because any behavior he performs other than teasing his sister is negatively reinforced by the absence of a scolding. If correct, this means that punishment procedures are actually a form of negative reinforcement. As with conditioned suppression theory, the avoidance theory of punishment assumes that punishment does not directly weaken a behavior. It simply replaces the punished behavior with an avoidance response of some sort. A disadvantage of this theory, however, is that it carries with it all of the theoretical difficulties associated with avoidance conditioning, some of which we discussed earlier in this chapter. The Premack Approach to Punishment As you will recall from Chapter!7, the Premack principle holds that a high-probability behavior (HPB) can be used to reinforce a low-probability behavior (LPB). As it turns out, the opposite can be applied to punishment. According to the Premack principle of punishment, an LPB can be used to punish an HPB (Premack, 1971a). Take, for example, a rat that is both hungry and fatigued from exercising. The rat in this condition is much more likely to eat food (an HPB) than to run in a wheel (an LPB). In terms of the Premack principle of reinforcement, this means that the behavior of eating food can be used as a reinforcer for the behavior of running in a wheel: Running in a wheel [LPB] (R) ! Eating food [HPB] (SR) Conversely, according to the Premack principle of punishment, one can also use running in a wheel to punish the behavior of eating: Eating food [HPB] (R) ! Running in a wheel [LPB] (SP) Thus, if eating food is followed by the consequence of being forced to run in a motorized wheel, the rat will be less likely to eat than if this consequence did not exist. To bring this point home, imagine how much easier it would be for a person who hates exercising to stick to a diet if they were forced to run a mile each time they ate something not on the diet. Note that this approach implicitly assumes that punishment is the opposite of reinforcement: If reinforcement strengthens behavior, punishment weakens behavior. In this sense, it differs from the previous two theories in that it views punishment as the mirror opposite of reinforcement. (Refer to Domjan, 2015, for an extended discussion of theories of punishment.) 353 Quick Quiz K 354 Chapter 9 Escape, Avoidance, and Punishment 1. According to the conditioned suppression theory of punishment, the application 2. 3. 4. 5. a behavior; instead, of punishment does not directly w reaction that tends to interfere with it produces an em ongoing behavior. This theory was based on evidence by Skinner that punishment tends to produce a (temporary/permanent) effect. This effect, however, probably results from using relatively (strong/weak) forms of punishment. According to the theory of punishment, a rat stops lever pressing when lever pressing is followed by shock because the occurrence of any behavior other than lever pressing is n by the nonoccurrence of shock. r According to the punishment version of the Premack principle, the occurrence behavior can be used to of a punish the occurrence of a behavior. This means that if Sally rarely washes dishes and often bites her nails, then can be used the behavior of . to punish the occurrence of According to the Premack principle, if running (an HPB) is followed by pushups (an LPB), then running should (decrease/increase) in frequency, which is an instance of (reinforcement/punishment). If push-ups (LPB) are followed by in frequency, running (HPB), then the push-ups should . which is an instance of 9.3 Noncontingent Punishment In the typical escape/avoidance procedure, the aversive stimulus is controllable in the sense that the animal is able to make a response that significantly reduces its effect. Likewise, in a punishment procedure, the animal has some semblance of control because if it does not make the response, it will not be punished. In both cases, some type of contingency exists. But what if such a contingency were absent? What if the aversive event was essentially uncontrollable (and even unpredictable), such that whatever you do, you are unable to influence your exposure to that event? In the same manner that noncontingent reinforcement has some unique effects on behavior (as discussed in Chapter 7), so too does noncontingent punishment. Let us examine some of these effects. Learned Helplessness Consider the following experiment by Seligman and Maier (1967). The experiment began with an initial phase in which dogs were suspended in a harness and exposed to one of three conditions. In an inescapable-shock condition, the dogs received a series of shocks but were unable to do anything about them. In an escapable-shock condition, the dogs received shocks but were able to terminate each shock by pressing a panel with their snout. Each dog in this condition was also yoked to (paired up with) a dog in the first 9.3 Noncontingent Punishment condition, such that when it turned off the shock for itself, it also turned off the shock for its partner dog in the other condition. Thus, the only difference between these two dogs was that the dog in the escapable-shock condition had control over the shocks and the dog in the inescapable-shock condition did not. Finally, some dogs were in a no-shock control condition: these dogs were never shocked and simply waited out the session suspended in the harness. In the next phase of the experiment, all of the dogs were exposed to a shuttle avoidance procedure in which the task was to learn to avoid shock by jumping over a barrier, each shock being preceded by a 10-second period of darkness. The dogs exposed to the no-shock control condition in the initial phase of the experiment soon learned to avoid the shock by jumping over the barrier during the period of darkness. The dogs exposed to the escapable-shock condition also learned the avoidance task quickly. The dogs from the inescapable-shock condition, however, behaved quite differently. When shocked, many of them initially ran around in great distress but then lay on the floor and whimpered. They made no effort to escape the shock. Even stranger, the few dogs that did by chance jump over the barrier, successfully escaping the shock, seemed unable to learn from this experience and failed to repeat it on the next trial. In summary, the prior exposure to inescapable shock seemed to impair the dogs’ ability to learn to escape shock when escape later became possible. This phenomenon is known as learned helplessness, a decrement in learning ability that results from repeated exposure to uncontrollable aversive events. Seligman and Maier (1967) theorized that the dogs became helpless because they had learned during exposure to inescapable shock that any attempt to escape was useless—in other words, they learned that there was a lack of contingency between making a response and achieving a certain outcome. As a result, when confronted with shock in a new situation, they simply gave up. Other researchers, however, have proposed alternative explanations. For example, one theory suggests that animals exposed to inescapable aversive stimulation are distressed, and because of this distress they have difficulty attending to the relationship between behavior and its outcomes. Evidence for this theory includes the fact that if animals are given a very salient feedback stimulus whenever they make a successful escape response, such as by sounding a loud bell, the learned helplessness effect may disappear and the animals may once more learn such tasks effectively (Maier, Jackson, & Tomie, 1987). Learned helplessness may account for certain difficulties experienced by humans. For example, Dweck and Reppucci (1973) found that children who attempted to answer unsolvable problems later had considerable difficulty answering solvable problems. This suggests that children who have difficulty passing math exams in school, possibly because of poor teaching, might grow up to become “math-anxious” individuals who quickly give up when confronted by any sort of math problem. Learned helplessness may even be implicated in certain forms of depression (Seligman, 1975). People who 355 356 Chapter 9 Escape, Avoidance, and Punishment Quick Quiz L suffer a series of uncontrollable aversive events—loss of a job, physical illness, divorce, and so on—may become extremely passive and despondent. Like animals exposed to inescapable shock, they show little interest in improving their lot in life. (Read also the opening vignette to this chapter.) Fortunately, researchers have discovered a way to eliminate learned helplessness. The helpless animal will eventually recover its ability to escape on its own if it is repeatedly forced to escape the aversive stimulus—for example, if it is repeatedly dragged from the shock side of the chamber to the no-shock side (Seligman & Maier, 1967; Seligman, Rosellini, & Kozak, 1975). In similar fashion, behavioral treatments for depression often involve encouraging the patient to accomplish a graded series of tasks, starting with relatively minor tasks, such as writing a letter, and progressing to more difficult tasks, such as seeking a new job (Seligman, 1975). Research has also suggested a means for preventing the development of learned helplessness. Experiments have revealed that prior exposure to escapable shock often immunizes an animal against becoming helpless when it is later exposed to inescapable shock (Seligman et al., 1975); the animal will persist in trying to escape the shock rather than give up. This suggests that a history of successfully overcoming minor adversities might immunize a person against depression when the person is later confronted by more serious difficulties. As a tree is strengthened by exposure to winds strong enough to bend but not break its limbs, so too individuals seem to be strengthened by exposure to manageable amounts of misfortune. 1. The original experiments on learned ____________________ revealed that dogs that had first been exposed to inescapable shock had (no difficulty/difficulty) learning an escape response when later exposed to (escapable/inescapable) shock. 2. It seemed as though these dogs had learned that there (is/is not) a contingency between their behavior and the offset of shock. 3. This effect can be overcome by (forcing/enticing) the dogs to make an escape response. As well, dogs that have had previous exposure to escapable shock are (more/less) susceptible to becoming helpless when later exposed to inescapable shock. 4. Learned helplessness may account for various difficulties in humans, including the clinical disorder known as d____________________. Masserman’s Experimental Neurosis As you may recall, experimental neurosis is an experimentally produced disorder in which animals exposed to unpredictable events develop neurotic-like symptoms. We first encountered this phenomenon in our discussion of Pavlov’s work on discrimination training in dogs (refer to Chapter 4). He and his colleagues discovered that dogs that had difficulty discriminating which cues predicted the delivery of food seemed to experience a nervous breakdown. Pavlov hypothesized that human neuroses might likewise develop as a result of exposure to unpredictable events. 9.3 Noncontingent Punishment A variation on Pavlov’s procedure, involving the use of aversive rather than appetitive stimuli, was developed by Masserman (1943). He found that cats that experienced unpredictable electric shocks or blasts of air while eating often developed a pattern of neurotic-like symptoms. For example, normally quiet cats became restless and agitated, and normally active cats became withdrawn and passive—sometimes even to the point of becoming rigidly immobile (a symptom known as catalepsy). The cats also developed phobic responses to cues associated with feeding (since feeding had become associated with shock), as well as unusual “counterphobic” responses (e.g., a cat might run to the goal box, stick its head inside the box, and then simply stare at the experimenter—as though to say, “let’s get it over with”). It generally took only two or three presentations of the aversive stimulus to elicit these symptoms, which might last several months. More recent work (but with rats, not cats) has shown that many of these symptoms are similar to those found in posttraumatic stress disorder (PTSD) in humans (e.g., Foa, Zinbarg, & Rothbaum, 1992). PTSD is a disorder that results from exposure to unpredictable life-threatening events, such as tornadoes, physical and sexual assaults, and battlefield experiences. Symptoms include sleep difficulties, exaggerated startle response, intrusive recollections about the trauma, and nightmares. Survivors of trauma often demonstrate fear and avoidance of trauma-associated stimuli (phobias) as well as a general numbing of responsiveness (e.g., a restricted range of emotions). Although the subjective symptoms of PTSD, such as intrusive recollections, are impossible to replicate in animals (we have no idea what animals are actually thinking), many of the overt symptoms, such as phobic behavior, agitation, and passivity, are similar to those shown by animals subjected to noncontingent, unpredictable aversive stimulation. Experimental neurosis is therefore proving to be a useful means for investigating the development of traumatic symptoms. For instance, as a general rule, traumatic symptoms are more easily induced in animals when the aversive stimulus is delivered in an environment that the animal has long associated with safety or some type of appetitive event. For example, unpredictable shocks delivered in a setting in which the animal typically eats food are especially likely to induce neurotic symptoms (Masserman, 1943). This suggests, for example, that symptoms of PTSD are more likely to arise when a person is unexpectedly attacked in the safety of their own home as opposed to a strange or dangerous area of town. The person who is attacked at home generalizes the experience and perceives the world as a dangerous, unpredictable place, with the result that they thereafter remain constantly vigilant (Foa et al., 1992). You may be wondering how Masserman’s experimental neurosis procedure differs from learned helplessness. The basic difference is that the typical learned helplessness procedure involves repeated exposure to aversive events that are predictable but uncontrollable. It is equivalent to being beaten up every day at 8:00 ".#. At first you try to escape from the beating, but 357 358 Chapter 9 Escape, Avoidance, and Punishment Quick Quiz M eventually you give up any hope of escape. Masserman’s experimental neurosis, on the other hand, involves infrequent but unpredictable exposure to aversive events. It is analogous to being unexpectedly dragged off every once in a while and beaten. The result is constant hypervigilance and an array of psychological and behavioral symptoms. But note that unpredictability also implies uncontrollability, so there is considerable overlap between the symptoms produced by learned helplessness and those produced by Masserman’s experimental neurosis procedure (Foa et al., 1992). 1. Experimental neurosis occurs when animals exposed to un____________________ events develop neurotic-like symptoms. 2. In these situations, normally (quiet/active) cats became restless and agitated, whereas normally (quiet/active) cats became withdrawn and passive. 3. When food was paired with unpredictable shock, the cats also developed p____________________ and counter____________________ responses to the food. 4. Evidence suggests that neurotic symptoms are more likely to develop when the traumatic event occurs in an environment that a person generally regards as (safe/dangerous). 5. Learned helplessness results from repeated exposure to aversive events that are ; experimental neurosis results from exposure to un events that are un____________________. Advice for the Lovelorn Dear Dr. Dee, I am in a relationship that is starting to depress me, though most of what is happening is quite subtle. For example, when I’m really excited about something, my partner will usually act quite disinterested. Similarly, when I suggest doing something that I think will be fun, she usually turns it down and suggests something else. She also gets snippy with me (or worse yet, gives me the silent treatment) when I am least expecting it. I have tried talking to her about it, but she says that I am overreacting and points to how affectionate she usually is (which is true). So Am I Overreacting? Dear So, It sounds like you’re in a relationship where much of your behavior is being subtly punished, some of it on a noncontingent basis. Thus, you’re starting to perceive that whatever you do will make little difference. It is therefore not surprising that you’re 9.3 Noncontingent Punishment 359 becoming depressed. You need to calmly point out to your partner the damaging effects of what she is doing, and the extent to which it is making you depressed. First, however, you might wish to examine your own behavior to see if you’re doing something to reinforce this pattern of behavior in your partner. Relationship problems are often a two-way street, with neither party solely responsible for the difficulty. For example, perhaps you acquiesce to your partner’s wishes so often that you are essentially reinforcing her for behaving this way. If that is the case, you may need to become a bit more assertive about your wishes. In fact, it could well be that she would be much happier if you were more assertive and your relationship with each other was more balanced. Behaviorally yours, And Furthermore Dissociative Identity Disorder: A Behavioral Perspective Some clinicians believe that the most severe disorder produced by trauma is dissociative identity disorder (DID) (formerly called multiple personality disorder). The essential characteristic of this disorder is two or more personality states (or alter personalities) that repeatedly take control of behavior. Patients also experience extensive amnesia, with some personalities often unaware of the existence of other personalities. For example, in the classic case of Eve White (portrayed in the 1957 movie The Three Faces of Eve), the original personality of Eve White was reportedly unaware of an alter personality named Eve Black. Eve Black, however, was fully aware of Eve White and enjoyed making life difficult for her (Thigpen & Cleckley, 1957). The concept of DID is, however, extremely controversial. Behaviorists have traditionally viewed multiple personalities as distinct patterns of behavior (both overt and covert) that have arisen in response to distinctly different contingencies of reinforcement (Skinner, 1953). This reasoning has been carried a step further in the posttraumatic model of DID, which assumes that DID usually results from childhood trauma (e.g., Ross, 1997). According to this model, an abused child can more easily cope with everyday life by usually forgetting about the abusive incidents and by pretending that the abuse is happening to someone else. In behavioral terms, this selfdeception can be conceptualized as a type of covert avoidance response—“Nothing bad is happening to me”—that is being negatively reinforced by a reduction in anxiety. As a result, the child learns to compartmentalize the distressing experience into a separate personality pattern that has its own dispositions and memories (Kohlenberg & Tsai, 1991). 360 Chapter 9 Escape, Avoidance, and Punishment This style of coping may become so habitual that it eventually results in the formation of dozens, or even hundreds, of separate personality states. Others, however, have argued that DID is usually not the result of trauma but instead the result of suggestive influence (Lilienfeld et al., 1999; Spanos, 1996). According to this sociocognitive model (which can also be conceptualized as a cognitive behavioral model), the patient’s displays of alter personalities have been inadvertently shaped through processes of social reinforcement and observational learning. Supportive evidence for this model includes the following: • The first clear observations of alter personalities are often obtained following exposure to a therapist who communicates to the patient that displays of alter personalities will be considered appropriate (and hence socially reinforced)—such as by asking the patient “if there is another thought process, part of the mind, part, person or force” within or who wishes to communicate with the therapist (e.g., Braun, 1980, p. 213). • The number of alter personalities displayed by patients usually increases as therapy progresses, as does the patients’ ability to quickly switch from one alter to another (Ross, 1997). This suggests that a process of shaping may be involved. • The number of DID cases rose sharply following dramatic presentations of the disorder to the public during the 1970s and 1980s, such as the case of Sybil, which became a best-selling book (Schreiber, 1973) and a popular movie. This suggests that observational learning may have played a role in the increased prevalence of the disorder. Direct evidence for the role of behavioral processes in DID was reported by Kohlenberg (1973). He found that by manipulating the amount of reinforcement a patient received for displaying one of three alter personalities, he was able to manipulate the amount of time a particular personality was displayed. He then devised a successful treatment program that included reinforcing displays of the personality that acted normally and ignoring displays of other personalities. Supporters of the posttraumatic model (e.g., Dalenberg et al., 2012; Gleaves, 1996; Ross, 1997; Ross & Norton, 1989) and the sociocognitive model (e.g., Lilienfeld et al., 1999; Lynn et al., 2014; Powell & Gee, 1999; Spanos, 1994, 1996) have each presented a series of arguments and counterarguments in support of their positions. The result has been some movement toward a middle ground. For example, Dalenberg et al. (2012) acknowledged that at least some cases of DID may have been artificially induced in therapy, while Lynn et al. (2014) acknowledged that a tendency toward developing DID-type symptoms might sometimes be the result of trauma. Likewise, Phelps (2000) has presented a behavioral account of DID, arguing that although alter personalities could conceivably arise from a history of childhood trauma, therapists might also inadvertently strengthen displays of alter personalities through processes of social reinforcement. Study Questions In Review To summarize this chapter, write two or three paragraphs that distinguish between escape and avoidance, describe Mowrer’s two-process theory, describe criticisms of two-process theory, and describe how avoidance conditioning interacts with phobic responses and obsessive–compulsive behavior patterns. Two or three more paragraphs could be devoted to comparing and contrasting the pros and cons of using both positive and negative punishment. In particular, you should be able to describe recommended practices for the use of punishment (including reinforcement of other behavior), and theories that help explain why punishment has undesirable effects. Finally, a paragraph or two can summarize laboratory examples of noncontingent punishment effects. Another way to review this chapter is to create personal examples of some of these concepts. Can you describe and diagram examples of situations where you experienced positive or negative punishment? Are there situations where you’ve experienced learned helplessness? Can you think of a situation in which you experienced punishment for a behavior but didn’t know what other behavior to do instead (and would have appreciated some reinforcement for an alternative behavior)? Further Exploration When people think of punishment, one of the first examples that may come to mind is spanking or other types of corporal punishment (“corporal” refers to the body). The American Psychological Association has a policy that recommends against the use of corporal punishment: https://www.apa .org/about/policy/corporal-punishment. You can also read more about research into the effects of spanking here: https://www.apa.org/monitor/2012/04/spanking. What about physical punishment for animals? There are negative effects for dogs, too, which you can read about here: https://www.psychologytoday .com/ca/blog/canine-corner/201205/is-punishment-effective-way-change -the-behavior-dogs. Here’s a real-life time-out procedure from an online video that went viral. Note especially the dad’s calm demeanor, clear explanations, and positive reinforcement of more appropriate behavior. What do you think of his approach? https://www.youtube.com/watch?v=X2SDsrd1lvo. Study Questions 1. Distinguish between escape and avoidance behavior and describe the evolution of these behaviors in a shuttle avoidance procedure. How does Mowrer’s two-process theory account for avoidance behavior? 361 362 Chapter 9 Escape, Avoidance, and Punishment 2. What are two criticisms of Mowrer’s two-process theory of avoidance? How have these criticisms been answered? 3. In what ways is experimental avoidance conditioning different from human phobic conditioning? According to Stampfl, what is a critical factor in the development and maintenance of phobic behavior that is often missing from experimental demonstrations of avoidance? 4. How can the two-process theory of avoidance account for obsessive– compulsive disorder? 5. Distinguish between time-out and response cost procedures and between extrinsic punishment and intrinsic punishment. Give an example of each. 6. What is the distinction between a primary punisher and a secondary punisher? What is a generalized punisher? Provide clear examples of each. 7. Briefly outline the problems listed concerning the use of punishment. 8. What is the major advantage of punishment over extinction? What are three beneficial side effects of punishment? 9. Briefly outline the characteristics of effective punishment. 10. Describe the conditioned suppression theory of punishment, the avoidance theory of punishment, and the Premack approach to punishment. 11. Describe the basic experimental procedure (with control group) that was first used to demonstrate learned helplessness in dogs and the outcome that was observed. How can learned helplessness in dogs be eliminated? How can dogs be immunized against the development of learned helplessness? 12. Describe Masserman’s procedure for inducing experimental neurosis in cats, and list some of the symptoms he observed. How does experimental neurosis differ from learned helplessness and how is it similar? Concept Review avoidance theory of punishment. The theory that punishment involves a type of avoidance conditioning in which the avoidance response consists of any behavior other than the behavior being punished. conditioned suppression theory of punishment. The theory that punishment does not weaken a behavior, but instead produces an emotional response that interferes with the occurrence of the behavior. exposure and response prevention (ERP). A method of treating obsessive– compulsive behavior that involves prolonged exposure to anxietyarousing events while not engaging in the compulsive behavior pattern that reduces the anxiety. extrinsic punishment. Punishment that is not an inherent aspect of the behavior being punished but is the result of a separate event that follows the behavior. Chapter Test generalized (or generalized secondary) punisher. An event that has become punishing because it has in the past been associated with many other punishers. intrinsic punishment. Punishment that is an inherent aspect of the behavior being punished, that is, the act of engaging in the behavior is itself punishing. learned helplessness. A decrement in learning ability that results from repeated exposure to uncontrollable aversive events. Premack principle of punishment. A low-probability behavior (LPB) can be used to punish a high-probability behavior (HPB). primary (or unconditioned) punisher. Any event that is innately punishing. response cost. A form of negative punishment involving the removal of a specific reinforcer following the occurrence of a behavior. secondary (or conditioned) punisher. An event that has become punishing because it has in the past been associated with some other punisher. time-out. A form of negative punishment involving the loss of access to positive reinforcers for a brief period of time following the occurrence of a problem behavior. two-process theory of avoidance. The theory that avoidance behavior is the result of two distinct processes: (1) classical conditioning, in which a fear response comes to be elicited by a CS; and (2) operant conditioning, in which moving away from the CS is negatively reinforced by a reduction in fear. Chapter Test 11. According to Mowrer, the two processes that underlie avoidance behavior are: (1) c conditioning, in which a(n) f response comes to be elicited by a CS; and (2) conditioning, in which moving away from the CS is reinforced by a reduction in . 3. According to the theory of punishment, if a rat is shocked for lever pressing, then any behavior other than will be reinforced by the nonoccurrence of shock. 27. If a father punishes his son for being aggressive with his playmates, the son may learn not to be aggressive only when the father is . 8. Otto woke up one night to find an intruder standing over him in his bedroom. When the intruder saw that Otto was awake, he stabbed him and fled. Boyd was walking through an unfamiliar part of town one night when he too was stabbed. In keeping with certain research findings on experimental , the person most likely to experience symptoms of PTSD is . 363 364 Chapter 9 Escape, Avoidance, and Punishment 14. One criticism of Mowrer’s two-process theory is that animals continue to make the avoidance response even though they no longer seem to be of the CS. One reply to this criticism is that although the animals may become significantly less ____________________ of the CS, they do not in fact become completely ____________________. 20. A person who checks their apartment door dozens of times each night to make sure that it is locked probably experiences a(n) in anxiety when they think about whether the door is locked and a(n) in anxiety when they check it. This acts as a reinforcer for the behavior of checking. 15. According to the theory of avoidance, I avoid bees simply because I am then less likely to be stung, not because I feel a reduction in fear. 9. Obert did not want to go to school one morning and pretended to be sick. Sure enough, his mother fell for the trick and let him stay home. Thereafter, Obert often pretended that he was ill so that he did not have to go to school. Obert’s tendency to pretend that he was sick was strengthened through the process of ____________________. 28. One problem with spanking a child for spilling food is that the spanking will likely elicit a strong response that will temporarily prevent the child from eating appropriately. The child may also become as a result of the spanking, which might later be directed toward his little brother or sister. He might also learn that an effective means of controlling others is through the use of . 19. Mowrer’s two-process theory seems highly applicable to obsessive– compulsive disorder in that the occurrence of an obsessive thought is associated with a(n) in anxiety, while performance of a compulsive behavior is associated with a(n) in anxiety. 2. Skinner concluded that punishment generates a conditioned reaction that suppresses any appetitive behavior, and that the appetitive behavior (will/will not) quickly recover one the punishment is withdrawn. Later research showed that this may be because Skinner had used a relatively (strong/weak) form of punishment in his research. 13. One criticism of Mowrer’s two-process theory is that an avoidance response often does not seem to , even after hundreds of trials. According to the hypothesis, however, this is because exposures to the aversive stimulus are too brief for to take place. Chapter Test 4. According to the Premack principle, if Rick smokes a lot and rarely vacuums, then can serve as an effective punisher for . 22. Losing your wallet by being careless is an example of a (negative/ positive) punisher, while getting a shock by being careless is an example of a punisher (assuming in each case that the behavior of carelessness subsequently in frequency). 12. According to Mowrer, I go out of my way to avoid bees because behaving this way has been (positively/negatively) (reinforced/punished) by a(n) in fear. 29. One problem with spanking a child for being noisy is that this will likely have a(n) effect in suppressing the behavior, which then serves as a for the increased use of spanking on future occasions. 25. If you spank a dog for making a mess on the carpet, the dog might learn to avoid rather than avoid making a mess on the carpet. 10. The theoretical difficulty with avoidance behavior, as opposed to escape behavior, is that the individual is moving from one (aversive/nonaversive) situation to another, and it is difficult to argue how a (change/lack of change) can serve as a reinforcer. 31. The beneficial side effects of punishment can include increases in!s behavior, improvements in m , and increased a to the environment. 21. One difference between OCD and a phobia is that a phobia usually involves a(n) (passive/active) avoidance response, while OCD usually involves a(n) avoidance response. 7. Pietro is having great difficulty sleeping, is easily startled, and has developed various phobias. Pietro’s symptoms are similar to those shown by Masserman’s cats that were exposed to aversive stimulation. This set of symptoms in experimental animals is known as experimental ; in humans, it is known as disorder. 16. Experiments with animals have shown that an effective way to overcome learned helplessness is to present a salient stimulus (like a loud noise) each time an escape response (occurs/fails to occur). 1. For children who are old enough to understand language, punishment should always be combined with a(n) . 23. Making a child sit in a corner for being too noisy is an example of a(n) procedure, while turning off the television set when the child is too noisy is an example of a(n) procedure. 365 366 Chapter 9 Escape, Avoidance, and Punishment 5. When Renee was in elementary school, she was cruelly teased by a classmate each recess. The teachers ignored her pleas for help, as did her classmates. Seligman would predict that as time passes, Renee is likely to (decrease/increase) her efforts to stop the teasing. In other words, she will begin to display learned . She may also become clinically . 17. According to Mineka, there are limitations in the extent to which experimental demonstrations of avoidance are analogous to human phobias. For example, in an experimental demonstration of avoidance that involves a tone and an aversive air blast, the rat will likely require (one/more than one) conditioning trial. By comparison, a bee phobia in humans typically requires conditioning trial. As well, the rat’s avoidance behavior is likely to be (more/less) consistent than the avoidance behavior in a bee phobia. 26. One problem with spanking a child for being noisy while playing with his friends is that he might not only stop being noisy but also stop . 30. A parent who wishes to punish their child’s behavior of playing too roughly with the cat would do well to impose the punishing consequence after the occurrence of the unwanted behavior and, at least initially, on a(n) (consistent/unpredictable) basis. The parent should also the behavior of playing appropriately with the cat. 6. According to learned helplessness research, Clint is (more/less) likely to become depressed following a divorce if his own parents divorced when he was a child and he successfully coped with the experience. 24. Hugh got injured at work while being reckless and as a result became less likely to act recklessly. Eduardo was reprimanded by the boss for being reckless and also became less likely to act recklessly. Getting injured is a (primary/secondary) punisher for the behavior of being reckless, while getting reprimanded is a punisher. 18. Stampfl demonstrated that a critical factor in phobic conditioning is the possibility of making a(n) (early/late) avoidance response, thereby minimizing the amount of involved in avoiding the feared event. Answers to Chapter Test 1. explanation 2. emotional; will; weak 3. avoidance; lever pressing; negatively 4. vacuuming; smoking 5. decrease; helplessness; depressed 6. less 7. unpredictable; neurosis; posttraumatic stress 8. neurosis; Otto 9. negative reinforcement 10. nonaversive; lack of change 11. classical; fear; operant; negatively; fear Answers to Chapter Test 12. negatively; reinforced; reduction 13. extinguish; anxiety conservation; extinction 14. fearful; fearful; nonfearful 15. one-process 16. occurs 17. more than one; one; less 18. early; effort 19. increase; decrease 20. increase; decrease; negative 21. passive; active 22. negative; positive; decreases 23. time-out; response cost 24. primary; secondary (or generalized secondary) 25. you 26. playing with his friends 27. present (or nearby) 28. emotional; aggressive (angry); punishment 29. immediate; negative reinforcer 30. immediately; consistent; positively reinforce 31. social; mood; attention 367 Chapter 10 Choice, Matching, and Self-Control Mark was becoming quite frustrated by Jon’s insistence that they were spending too much time together. He told Jon that if two people truly love each other, they should want to spend as much time together as possible. Jon countered that he did love Mark, but that spending too much time together was making their relationship dull and boring. For Jon, life was more fulfilling when he interacted with a variety of people each day. Chapter Outline 10.1 Choice and Matching Concurrent Schedules The Matching Law Deviations from Matching Matching and Melioration 10.2 Self-Control Mischel’s Delay of Gratification Paradigm The Ainslie–Rachlin Model of Self-Control The Small-But-Cumulative Effects Model Skinner on Self-Control Self-Control as a Temporal Issue Learning Objectives 10-1 10-2 10-3 10-4 10-5 368 Describe how concurrent schedules of reinforcement are used to study choice behaviors. Define the matching law. Describe undermatching, overmatching, and bias from matching, including predictions for when those deviations from matching will occur. Distinguish between matching and melioration. Outline the challenges of defining self-control and willpower. 10.1 10-6 10-7 10-8 10-9 Choice and Matching Describe Skinner’s views on self-control, including the distinction between controlling and controlled responses. Describe the effects of immediate and delayed reinforcement and punishment on self-control. Describe the Ainslie–Rachlin model of self-control, including predictions about when impulsive behavior is likely to occur and when commitments are likely to be made Describe several strategies to improve self-control. Operant conditioning in the real world is rarely a matter of being offered only one source of reinforcement. Instead, individuals typically choose between alternative sources of reinforcement. In this chapter, we examine some of the principles by which such choices are made—especially the principle of matching, which stipulates that the amount of behavior directed toward an alternative is often proportional to the amount of reinforcement we receive from that alternative. We also examine the types of choices involved when people attempt to exert “self-control” over their behavior. 10.1 Choice and Matching Concurrent Schedules In operant conditioning experiments, investigations of choice behavior often make use of a type of complex schedule known as a concurrent schedule. A concurrent schedule of reinforcement consists of the simultaneous presentation of two or more independent schedules, each leading to a reinforcer. The organism is thus allowed a choice between responding on one schedule versus the other. For example, a pigeon may be given a choice between responding on a red key that is associated with a VR 20 schedule of reinforcement and a green key that is associated with a VR 50 schedule of reinforcement (refer to Figure 10.1). We can diagram this situation as follows: Red key: Key peck VR 20 Food Green key: Key peck VR 50 Food Which alternative would you choose? If you think of this situation as analogous to choosing between two slot machines, one paying off after an average of 20 dollars are spent and the other paying off after an average of 50 dollars are spent, the choice becomes easy. You would pick the better-paying 369 370 Chapter 10 Choice, Matching, and Self-Control Illustration of a two-key operant procedure in which two schedules of reinforcement are simultaneously available: in this case, a VR 20 schedule on the red key and a VR 50 schedule on the green key. The two schedules thus form the two components of a concurrent VR 20 VR 50 schedule of reinforcement. (Source: Domjan, 2003.) Figure 10.1 Schedule A VR 20 Schedule B VR 50 Key A Key B Red Green machine: the one that requires an average of only 20 dollars to produce a win (if you can fight off everyone else who wants that machine). Similarly, the pigeon will come to develop an exclusive preference for the VR 20 alternative (Herrnstein & Loveland, 1975). Choice between concurrent VR schedules is easy because an exclusive preference for the richer alternative clearly provides the better payoff. But what about concurrent VI schedules? What if, for example, a pigeon is presented with a concurrent VI 30-sec VI 60-sec schedule? Red key: Key peck V1 30-sec Green key: Key peck V1 60-sec Food Food Remember that on VI schedules, reinforcers become available at unpredictable points in time (and any responses before that point will not result in reinforcement). Given this unpredictability, will the bird just randomly distribute its responses between the two alternatives, hoping to 10.1 Choice and Matching 371 catch the reinforcers on each alternative as they become available? (Just as in trying to phone two friends, you might repeatedly dial each number in random order hoping to catch each person as soon as they are available.) Herrnstein (1961) carried out just such an experiment using various schedule values and found that the pigeon’s behavior under such circumstances is actually quite systematic. It is so systematic, in fact, that it led to the formulation of what is known as the matching law. schedule of reinforcement, two or more available. schedules of reinforcement are s 2. In a choice between a VR 25 and a VR 75 schedule of reinforcement, your best strategy would be to choose the VR 25 schedule (100/50/25)% of the time. The Matching Law The matching law holds that the proportion of responses emitted on a particular schedule will match the proportion of reinforcers obtained on that schedule (note that it is proportion of responses and reinforcers and not number of responses and reinforcers). Thus, it predicts that a pigeon will emit approximately twice as many responses on a VI 30-sec schedule as on a VI 60-sec schedule because the rate of reinforcement on the former will be twice as great as on the latter (an average of two reinforcers per minute on the VI 30-sec schedule versus one reinforcer per minute on the VI 60-sec schedule). Similarly, a pigeon will emit three times as many responses on a VI 20-sec schedule as it will on a VI 60-sec schedule because the VI 20-sec schedule provides three times the rate of reinforcement (an average of three reinforcers per minute on the VI 20-sec schedule versus one per minute on the VI 60-sec schedule). The matching law therefore predicts a consistent relationship between the proportion (or percentage) of reinforcers obtained on a certain alternative and the proportion (or percentage) of responses emitted on that alternative. If a pigeon earns 10% of its reinforcers on an alternative, then it will likely have emitted about 10% of its responses on that alternative; if it earns 60% of its reinforcers on an alternative, then it will likely have emitted about 60% of its responses on it. The matching law can also be expressed in the form of an equation: RA RA 1 RB 5 SRA SRA 1 SRB This equation can also be written as follows, which will be easier to use if you are typing your notes or completing an online examination: RA/(RA + RB) = SRA/(SRA + SRB) Within this equation, R represents the number of responses emitted, SR is the number of reinforcers earned, and the subscripts A and B refer to the two schedules of reinforcement. Thus, RA is the number of responses emitted on Quick Quiz A 1. On a c 372 Chapter 10 Choice, Matching, and Self-Control schedule A, RB is the number of responses emitted on schedule B, SRA is the number of reinforcers earned on schedule A, and SRB is the number of reinforcers earned on schedule B. Therefore, the term to the left of the equal sign: RA/(RA + RB) indicates the proportion of responses emitted on schedule A. It is the number of responses emitted on schedule A divided by the total number emitted on both schedules. The term to the right of the equal sign: SRA/(SRA + SRB) indicates the proportion of reinforcers earned on schedule A. It is the number of reinforcers earned on schedule A divided by the total number earned on both schedules. To illustrate how the equation works, let’s look at some hypothetical data from an imaginary pigeon that we will name Cher Ami, after the famous WWI messenger pigeon credited with saving the lives of the “lost battalion.” (For more information about the real Cher Ami, see Further Exploration at the end of the chapter.) Imagine that Cher Ami is given a choice between responding on a VI 20-sec versus a VI 60-sec schedule of reinforcement. The first assumption one should make is that she will go back and forth between the two alternatives in order to pick up all or almost all of the available reinforcers on each schedule—which is in fact what hungry pigeons readily learn to do. This means that Cher Ami will earn about three times as many reinforcers on the VI 20-sec schedule (which provides an average of three reinforcers per minute) as on the VI 60-sec (which provides an average of one reinforcer per minute). Imagine that during a 60-minute session, this is precisely what she does: she earns all of the 180 reinforcers that became available on the VI 20-sec schedule during that one hour session and all of the 60 reinforcers that became available on the VI 60-sec schedule. If we insert these values into the right-hand (reinforcer) side of the equation, we get the following: SRVI 20s/(SRVI 20s + SRVI 60s) = 180/(180 + 60) = 180/240 = .75 This means that the proportion of reinforcers that Cher Ami obtained from the VI 20-sec schedule during that session was .75. In other words, 75% (or three-fourths) of the reinforcers that she earned were obtained from the VI 20-sec schedule, which also means that the remaining 25% (or one-fourth) of the reinforcers were obtained from the VI 60-sec schedule (100% – 75% = 25%). Therefore, as expected, Cher Ami obtained three times as many reinforcers from the VI 20-sec schedule as from the VI 60-sec schedule (75% vs. 25%). The question now is, did Cher Ami match the proportion of responses she emitted on each alternative to the proportion of reinforcers she obtained from each alternative? To answer this question, we have to look at the actual number of responses she emitted on each alternative. Suppose, for example, 10.1 Choice and Matching that during that one-hour session, she emitted 2,240 responses on the VI 20-sec schedule and 775 responses on the VI 60-sec schedule. She clearly emitted more responses on the richer schedule, but did she closely match proportion of responses to proportion of reinforcers obtained? To determine this, we need to insert those response numbers into the left-hand (response) side of the equation. When we do that, we get the following: RVI 20s /(RVI 20s + RVI 60s) = 2,240/(2,240 + 775) = 2,240/3,015 = .74 This shows that Cher Ami emitted .74 (or 74%) of her responses on the VI 20-sec schedule, which closely matches the proportion of reinforcers, .75 (or 75%), that she obtained on the VI 20-sec schedule. Thus, as predicted by the matching law, the proportion of responses that Cher Ami emitted on the VI 20-sec schedule closely matches the proportion of reinforcers that she obtained on that schedule. Note that she did not have to distribute her responses in this way in order to pick up all of the available reinforcers. For example, she could just as easily have obtained all of them by distributing her responses equally between the two alternatives. But she didn’t do that; instead, in keeping with the matching law, she closely matched the proportion of responses she emitted on an alternative to the proportion of reinforcers she obtained on that alternative. (For results from Herrnstein’s [1961] original matching experiment in which pigeons chose between several different combinations of schedules, refer to Figure 10.2.) Note: If asked to calculate what the matching law predicts concerning the expected proportion of responses emitted on two concurrent VI schedules of reinforcement, you should: (1) assume that the pigeon (or rat or person) will earn all of the available reinforcers on each schedule during the session, and (2) insert the expected number of reinforcers earned on each schedule into the right-hand (reinforcer) side of the matching equation. This will give you the expected proportion of reinforcers that will be earned on each schedule. According to the matching law, this proportion will also be the predicted proportion of responses emitted on each schedule. Note that you do not need to bother with the left-hand (response) side of the equation in making this prediction. You use that side of the equation only if you have already run such a session and have actual response numbers to work with; this will then enable you to confirm whether the pigeon did in fact match the proportion of responses emitted to the proportion of reinforcers earned on each alternative. Matching appears to be a basic principle of choice behavior, applicable to a variety of situations and species. For example, Houston (1986) investigated the extent to which the pied wagtail, an insectivorous bird in Britain, distributed its foraging behavior between two separate patches of food: (1) a stretch of territory along the banks of the Thames River, which the territorial owner defended from other wagtails (and only some birds owned territories), and (2) an open meadow that any wagtail could visit and feed upon as part of 373 Chapter 10 Choice, Matching, and Self-Control Experimental results depicting the proportion of responses emitted by two pigeons on key A. Different combinations of schedules were offered on key A versus key B across the different conditions of the experiment, with the schedule values ranging from VI 90-sec to VI 540-sec to extinction (no reinforcers available). As the schedule combinations changed and the proportion of reinforcers earned on key A increased from approximately .1 to 1.0, the proportion of responses emitted on key A increased in similar fashion. (Source: Adapted from “Relative and absolute strength of response as a function of frequen

Introduction to Learning and Behavior, 6th Ed. - Psychology Textbook

Related documents

Study collections

Products

Support

Introduction to Learning and Behavior, 6th Ed. - Psychology Textbook

Related documents

Study collections

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib